(@menu, @detailmenu): Remove entries about de-ANSI-fication
support.
+2011-08-29 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: better respect of TAP usage in wrapped tests
+ * tests/gen-wrap-tests: Partial rewrite to take into account the
+ possibility that the wrapped test uses the TAP protocol. This
+ complicates the code quite a bit, but is essential to ensure real
+ correctness in all the wrapped scripts.
+ * tests/wrap-tests.sh: Update, and add few more sanity checks.
+
+2011-08-29 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: use both perl and awk implementations for tests on TAP
+ * tests/gen-wrap-tests (get_list_for_pltap): New function, gets
+ the list of tests that check the Automake TAP support.
+ ($defines_for_pltap): New variable.
+ ($wrapper_type_list): Add "pltap".
+ * tests/defs-static.in: Check that `$am_tap_implementation' is
+ not exported in the environment.
+ * tests/self-check-env-sanitize.tap: Update.
+
+2011-08-29 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: two single auxiliary scripts to run/generate wrapped tests
+ * tests/config-shell-tests.sh, tests/parallel-tests.sh,
+ tests/gen-config-shell-tests, tests/gen-parallel-tests: Deleted,
+ their roles taken over by ...
+ * tests/wrap-tests.sh, tests/gen-wrap-tests: ... these new more
+ general scripts.
+ * tests/Makefile.am (EXTRA_DIST): Distribute the new scripts, and
+ do not distribute the obsolete ones anymore.
+ ($(srcdir)/parallel-tests.am, $(srcdir)/config-shell-tests.am): Do
+ not include nor generate anymore, they have been superseded by ...
+ ($(srcdir)/wrap-tests.am): ... this new fragment, automatically
+ generated by `gen-wrap-tests'.
+ (PTEST_LOG_COMPILER, SHTST_LOG_COMPILER): Delete, they are now
+ defined in the new `wrap-tests.am' makefile fragment.
+ (TEST_EXTENSIONS): Do not append to it anymore, that is now done
+ by the new `wrap-tests.am' makefile fragment above.
+ ($(parallel_tests:.ptest=.log), $(config_shell_tests:.shtst=.log),
+ $(parallel_tests), $(config_shell_tests): Remove this dependencies,
+ superseded by ...
+ ($(wrap_TESTS), $(wrap_LOGS)): ... these ones.
+ (MAINTAINERCLEANFILES): Delete, it's not used anymore.
+ (TESTS): Do not contain anymore `$(config_shell_tests)' nor
+ `$(parallel_tests)', contain `$(wrap_TESTS)' instead, as defined
+ by `$(srcdir)/wrap-tests.am'.
+ Other related minor edits and reorderings.
+ * bootstrap, tests/.gitignore, tests/README: Update.
+
+2011-08-29 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: list of tests placed in a separate Makefile fragment
+ This change allows us to easily use make instead of by-hand (and
+ more fragile) grepping to extract the list of hand-written tests
+ from our Makefiles.
+ * bootstrap (BOOTSTRAP_MAKE): New variable.
+ Use it to run make in order to extract the list of hand-written
+ tests from ...
+ * tests/list-of-tests.mk: ... this new file, where they are
+ listed in ...
+ (handwritten_TESTS, tap_with_common_setup_TESTS, tap_other_TESTS):
+ ... these variables, extracted respectively from ...
+ * tests/Makefile.am (handwritten_tests, tap_other_tests,
+ tap_with_common_setup_tests): ... these older variables, now
+ removed.
+ (EXTRA_DIST, expected_list_of_tests): Update w.r.t. the variable
+ names' changes.
+ (include $(srcdir)/list-of-tests.mk): New inclusion.
+ Remove an overkill indirection, and with it ...
+ (tap_with_common_setup_logs): ... this variable.
+ ($(srcdir)/parallel-tests.am, $(srcdir)/config-shell-tests.am):
+ Update to match the new interface of ...
+ (gen-parallel-tests, gen-config-shell-tests): ... these scripts,
+ that now read the list of tests from the standard input.
+
+2011-08-29 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: prefer AM_... variables to user-reserved ones in Makefile
+ * tests/Makefile.am (TAP_LOG_DRIVER_FLAGS): Renamed ...
+ (AM_TAP_LOG_DRIVER_FLAGS): ... to this.
+
+2011-08-29 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: some minor fixlets w.r.t. TAP-based tests
+ The variable that is used by our test scripts to decide whether
+ they are "plain" or TAP-based is `using_tap'; however, some
+ peripheral sanity checks and sanitization code in our testsuite
+ were erroneously using the `use_tap' variable instead. Fix that.
+ * tests/Makefile.am (AM_TESTS_ENVIRONMENT): Nullify `using_tap',
+ not `use_tap'.
+ * tests/defs-static.in: Check that `using_tap', not `use_tap',
+ is not exported.
+ * tests/self-check-env-sanitize.test: Update.
+
+2011-08-29 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: avoid re-running a test with the 'parallel-tests' option
+ * tests/backcompat5.test (parallel_tests): Define to "no", to
+ prevent the generation of a sibling test script that uses the
+ 'parallel-tests' driver. That would be useless anyway, since
+ we manually override the contents of configure.in, thus clobbering
+ the `parallel-tests' option in AM_INIT_AUTOMAKE.
+ Fix a typo in comments since we are at it.
+
+2011-08-29 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: defaults to awk+shell driver in tests on TAP support
+ * tests/defs ($am_tap_implementation): Default to "shell", not
+ to "perl".
+
2011-08-25 Stefano Lattarini <stefano.lattarini@gmail.com>
tests: list "forgotten" test script in TESTS
* lib/ylwrap: Convert tabs to spaces.
Reported by Karl Berry.
+2011-08-28 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: different test scripts cannot share the same basename
+ If two test scripts have the same basename, they will end up
+ sharing the same log file, leading to all sort of undefined and
+ undesired behaviours. Add a Makefile check verifying that this
+ is not the case.
+ * tests/test-logs-repeated.test: New test, verifying previously
+ unspecified semantics (in the definition of $(TEST_LOGS) from
+ $(TESTS)) that we use in our new check.
+ * tests/Makefile.am (TESTS): Update.
+ (check-no-repeated-test-name): New .PHONY check.
+ (check-local): Depend on it.
+
+2011-08-26 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: fix spurious failure for awk implementation of TAP driver
+ * tests/tap-realtime.test: The awk+shell implementation of the TAP
+ driver must "read ahead" of one line in order to catch the exit
+ status of the test script it runs. So, in the TAP-producer script
+ here, be sure to echo one "dummy" line after each TAP result line
+ in order not to cause false positives.
+
+2011-08-25 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap/awk: allow escaping of TAP directives
+ * lib/tap-driver.sh (setup_result_obj): Handle escaping of TAP
+ directives in a way tat is (mostly) compatible by what is done
+ by the TAP::Parser module.
+ With this change, the tests `tap-escape-directive.test' and
+ `tap-escape-directive-2.test' now also pass with the shell/awk
+ implementation of the TAP driver.
+
+2011-08-25 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ coverage: more about escaping of TAP directives
+ * tests/tap-escape-directive.test: Make grepping of testsuite
+ progress output slightly stricter.
+ * tests/tap-escape-directive-2.test: New test (failing with the
+ awk implementation of the TAP driver).
+ * tests/Makefile.am (tap_with_common_setup_tests): Update.
+
+2011-08-25 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap: improve syncing between awk+shell and perl implementations
+ * lib/tap-driver.pl (stringify_test_result): Renamed ...
+ (stringify_result_obj): ... to this. Break up a clause in the
+ long "if/elsif/.../else" construct to avoid unaesthetic line
+ breaks and to be more synced with the sibling function in
+ `tap-driver.sh'. Rename the `$result', `$PASS' and `$FAIL'
+ variables to respectively `$result_obj', `$COOKED_PASS' and
+ `$COOKED_FAIL', for clarity and better syncing.
+ (handle_tap_test): Renamed ...
+ (handle_tap_result): ... to this, and change the name of the
+ `$test' local variable to `$result_obj'.
+ (extract_comment): Reimplement using the simpler `index' and
+ `substr' builtins, rather than with more advanced uses of
+ regular expressions.
+ (%test_results, @test_results): Renamed respectively ...
+ (%test_results_seen, @test_results_list): ... to these, and
+ related adjustments throughout the `TEST_RESULTS' block.
+ (main, get_global_test_result): Refactor and do some cosmetic
+ changes to make these functions clearer and better synced with
+ sibling code in `tap-driver.sh'.
+ Other minor cosmetic and typo fixes.
+ * lib/tap-driver.sh (extract_tap_comment): Remove outdated
+ "FIXME" comments.
+ (get_global_test_result): Small reordering to make it better
+ synced with its sibling function in `tap-driver.pl'.
+ (stringify_result_obj): Consistently use `result_obj' as the
+ parameter name.
+ Other minor cosmetic and typo fixes.
+
+2011-08-25 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap/perl: don't redirect perl warnings/errors to log files
+ With this change, the test `tap-driver-stderr.test' also passes
+ with the perl implementation of the TAP driver.
+ * lib/tap-driver.pl (start): Save the original stderr into the
+ `OLDERR' file handle, and call ...
+ (trap_perl_warnings_and_errors): ... this new function, trapping
+ the `__WARN__' and `__DIE__' "pseudo-signals" to ensure that the
+ warning and error messages goes to that original stderr stream.
+ Since we are at it, be sure to prepend all possible "die" message
+ with the name of the script as given by the `$ME' variable.
+
+2011-08-25 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap/perl: add copyright notice, version string, and emacs stuff
+ * lib/tap-driver.pl: Add proper copyright notice and bug reporting
+ address. Remove obsolete heading comments. Add trailing comments
+ for emacs w.r.t. code-formatting rules and automatic updating of
+ the date-based script version saved into the `$VERSION' variable.
+ ($VERSION): Define to the current UTC date.
+
+2011-08-24 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap/awk: don't redirect awk stderr to log files
+ * lib/tap-driver.sh: Handle the file descriptors redirections
+ more smartly, to avoid sending error messages from awk (about
+ e.g. syntax or I/O errors) to the log files instead that to the
+ console.
+ * tests/tap-driver-stderr.test: New test, verifying the improved
+ driver behaviour. Notice that this test still fails when using
+ the perl implementation of the TAP driver.
+ * tests/Makefile.am (tap_other_tests): Update.
+
+2011-08-24 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap/awk: remove obsolete "FIXME" comment
+ * lib/tap-driver.sh: Remove obsolete "FIXME" comment about our
+ driver losing the exit status of the tested program; this issue
+ has been solved in precedent commit `v1.11-1052-gd630a0d'. Fix
+ an unrelated typo in comments since we are at it.
+
+2011-08-22 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: avoid spurious failure on Solaris
+ * tests/tap-signal.test (signal_caught): Fix a regression from
+ commit `v1.11-1052-gd630a0d' that was causing this test to fail
+ spuriously on Solaris (due to an already worked-around Solaris
+ /bin/sh bug, see commit `v1.11-981-g0ef3ef5').
+
+2011-08-22 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: use the new awk+shell TAP driver in our own testsuite
+ * tests/Makefile.am (TAP_LOG_DRIVER): Redefine to use the new
+ awk+shell tap driver, which should be complete enough for use in
+ a "controlled" production environment.
+
+2011-08-22 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap/awk: handle non-zero exit status from the test command
+ * lib/tap-driver.sh (get_test_exit_message): New function in the
+ awk script, used to extract the exit status of the test program,
+ or at least a good guess of it.
+ (write_test_results): Use it, reporting an ERROR result if it
+ detects that the test program exited with a non-zero status.
+ * tests/tap-signal.test: Account for the differences in the
+ error messages generated by the awk and perl TAP drivers. Fix
+ an unrelated typo in comments since we are at it.
+
+2011-08-22 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap/awk: prepare to fetch the exit status of the test command
+ * lib/tap-driver.sh: Rewrite some logic in the main parsing loop,
+ to make it possible to read the exit status of the test command
+ from the last line of the input stream.
+
+2011-08-22 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap/awk: refactor for future changes
+ * lib/tap-driver.sh: Rewrite the awk script so that the parsing
+ of the input stream is implemented as a hand-rolled loop in the
+ BEGIN block, using the `getline' builtin.
+
+2011-08-22 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: avoid spurious failure if rst2html program is missing
+ * tests/tap-html.test ($required): Require `rst2html'.
+
+2011-08-22 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap/awk: support Solaris /usr/xpg4/bin/awk
+ * lib/tap-driver.sh: Use `\t', not `\\t', to represent tabulation
+ character in regexps, even when inside double-quoted strings; that
+ seems to be more portable to Solaris 10 XPG4 awk.
+
+2011-08-22 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap/awk: support Solaris /usr/bin/nawk
+ * lib/tap-driver.sh (close_or_die): Remove, the use of the `close'
+ builtin in there was confusing /usr/bin/nawk from Solaris 10 into
+ a syntax error.
+ (write_test_results): Simply use `close' on the `*.trs' file, that
+ should be enough.
+
+2011-08-21 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap/awk: correctly extract message from SKIP plans
+ * lib/tap-driver.sh (/^1\.\.0+[ \t]*#/): Add a forgotten `?' to
+ the regex used to extract the skip message from a "1..0" TAP plan.
+ * tests/planskip.test: Check for the just-fixed bug.
+ * tests/tap-planskip-unplanned-corner.test: Don't be too strict
+ w.r.t. the formatting of the "skip whole test" message, to avoid
+ failing due to bugs irrelevant for this test (like the one just
+ fixed).
+
+2011-08-21 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap/awk: support colorized output
+ * lib/tap-driver.sh: Pass definition of colors to the BEGIN block
+ of the awk script, using ...
+ ($init_colors): ... this new variable, which supersedes ...
+ ($red, $grn, $lgn, $blu, $mgn, $std): ... these variables, which
+ have been consequently removed.
+ ($color_tests): We don't have to pass this anymore to the awk
+ script, so do not convert it to a numeric boolean.
+ (decorate_result): Implement for real.
+ With this change, the test 'tap-color.test' now passes.
+
+2011-08-21 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap: real (but still incomplete) awk implementation of TAP driver
+ * lib/tap-driver.sh: Add an incomplete, but mostly working,
+ implementation of a TAP parser and driver in awk. It doesn't yet
+ support colorized output, fetching of exit status from test
+ programs, nor a way to escape TAP directives in TAP result lines,
+ but passes all the tests of TAP support in the automake testsuite,
+ apart from the following ones:
+ - tap-color.test
+ - tap-escape-directive.test
+ - tap-exit.test
+ - tap-missing-plan-and-bad-exit.test
+ - tap-passthrough-exit.test
+ - tap-planskip-badexit.test
+ - tap-planskip-unplanned-corner.test
+ - tap-signal.test
+ Tested on Debian GNU/Linux with GNU awk 3.1.7 and 3.0.2, "original
+ awk" 2010-05-23-1, and mawk 1.3.3-15 (with which also the test
+ `tap-realtime.test' fails, in addition to those listed above).
+
+2011-08-21 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: fix spurious failure due to Solaris 'wc' incompatibility
+ * tests/tap-number-wordboundary.test ($planned): Strip any extra
+ whitespace from the output of `wc', to cater to Solaris 10.
+
+2011-08-21 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ coverage: keyword "SKIP" in TAP plan is case-insensitive
+ * tests/tap-planskip-case-insensitive.test: New test.
+ * tests/tap-planskip.test: Remove now-duplicated checks.
+ * tests/Makefile.am (tap_with_common_setup_tests): Update.
+
+2011-08-21 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ coverage: ambiguous use of TAP "TODO"/"SKIP" directives
+ * tests/tap-ambiguous-directive.test: New test.
+ * tests/tap-todo-skip-together.test: Reference it in heading
+ comments.
+ * tests/Makefile.am (tap_with_common_setup_tests): Update.
+
+2011-08-21 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ coverage: non-directive comments in TAP results are preserved
+ * tests/tap-result-comment.test: New test.
+ * tests/Makefile.am (tap_with_common_setup_tests): Update.
+
+2011-08-20 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ coverage: TAP result number followed by non-whitespace word boundary
+ * tests/tap-number-wordboundary.test: New test.
+ * tests/Makefile.am (tap_with_common_setup_tests): Update.
+
+2011-08-20 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ coverage: a TAP result numbered "0" is always out-of-order
+ * tests/tap-test-number-0.test: New, checks that a test result
+ numbered as 0 is to be considered out-of-order; this is
+ consistent with the behaviour of the `prove' utility.
+ * tests/Makefile.am (tap_with_common_setup_tests): Update.
+
+2011-08-20 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ cosmetics: fix botched indentation in perl TAP driver
+ * lib/tap-driver.pl (main, Getopt::Long::GetOptions): Fix
+ improper or botched indentation.
+
+2011-08-20 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ coverage: some more tests on corner cases of TAP support
+ * tests/tap-no-spurious.test: Extend checks verifying that a line
+ matching, say, the regex "^ok[a-zA-Z0-9_]" is *not* considered a
+ TAP result.
+ * tests/tap-no-spurious-numbers.test: New test, checking that our
+ TAP driver doesn't spuriously recognize as TAP result numbers what
+ is not (even if it seems pretty close).
+ * tests/tap-negative-numbers.test: New test, checking that our
+ TAP driver doesn't spuriously recognize negative TAP result
+ numbers, but correctly interprets them as test descriptions
+ instead.
+ * tests/tap-plan-leading-zero.test, tap-numbers-leading-zero.test:
+ New tests, checking how our driver fares in recognizing numbers
+ with leading zeroes in TAP results or TAP plans.
+ * tests/tap-planskip-malformed.test: New test, checking that a
+ malformed TAP plan is not recognized as a valid plan.
+ * tests/tap-plan-whitespace.test: New test, checking that a TAP
+ plan line with trailing whitespace is recognized and handled
+ correctly.
+ * tests/Makefile.am (tap_with_common_setup_tests): Update.
+
+2011-08-19 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: various fixlets and tweakings
+ * tests/seenc.test: Make grepping of automake stderr stricter.
+ Add a trailing `:' command.
+ * tests/symlink.test: Likewise. Also, define `$am_create_testdir'
+ to "empty" to avoid bringing in unused auxiliary files (only to
+ have to remove them right away), and use proper m4 quoting in
+ `configure.in'.
+ * tests/vpath.test: Make grepping of generated `Makefile.in'
+ slightly stricter. Prefer trailing `:' over trailing `Exit 0'.
+ * tests/suffix6c.test: Unset OBJEXT to avoid interferences from
+ the environment.
+ * tests/suffix12.test: Do not whitespace-indent `##' comments
+ when they are embedded in a makefile rule: having them indented
+ is not part of the Automake API, and might cause failures with
+ e.g., Tru64 make.
+ * tests/syntax.test: Simplify the `Makefile.am' to ensure that
+ automake doesn't fail for the wrong reasons. Make grepping of
+ automake stderr slightly stricter.
+ * tests/test-harness-vpath-rewrite.test: Remove useless variable
+ definition from `Makefile.am'.
+ * tests/test-driver-custom-multitest.test: Fix typo in comments.
+ * tests/self-check-me.tap: Fix minor typo in test description.
+ * tests/vars3.test: Make grepping of automake stderr stricter
+ and safer.
+ * tests/version6.test: Add sanity check, verifying that the
+ version number extracted from `automake --version' output seems
+ legit.
+ * tests/auxdir2.test: Renamed ...
+ * tests/auxdir-compauted.tap: ... to this, and converted to the
+ use of TAP.
+ * tests/auxdir4.test: Renamed ...
+ * tests/auxdir-unportable.tap: ... to this, and converted to the
+ use of TAP.
+ * tests/auxdir3.test: Renamed ...
+ * tests/auxdir-misplaced.test: ... to this.
+ * tests/auxdir5.test: Renamed ...
+ * tests/auxdir-nonexistent.test: ... to this.
+ * tests/auxdir9.test: Renamed ...
+ * tests/auxdir-autodetect.test: ... to this.
+ * tests/Makefile.am (TESTS): Update.
+ (XFAIL_TESTS): Remove `auxdir2.test'.
+
+2011-08-19 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: fix some redundant autotools calls in tests on TAP
+ * tests/tap-deps.test: Do not call autotools and configure; there
+ is no need to, since this script sources `tap-setup.sh'.
+ * tests/tap-empty-diagnostic.test: Likewise.
+ * tests/tap-escape-directive.test: Likewise.
+ * tests/tap-fancy2.test: Likewise.
+ * tests/tap-no-spurious.test: Likewise.
+ * tests/tap-out-of-order.test: Likewise.
+
2011-08-17 Stefano Lattarini <stefano.lattarini@gmail.com>
* THANKS (Daniel Richard G.): Update e-mail address.
* HACKING ("Working in git"): Use autoconf and autom4te 2.68,
not 2.67, in the examples.
+2011-08-17 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap: improve granularity for tests on problematic TAP messages
+ * tests/tap-message-0.test: Break up into ...
+ * tests/tap-msg0-result.test, tests/tap-msg0-directive.test,
+ tests/tap-msg0-planskip.test, tests/tap-msg0-misc.test,
+ tests/tap-msg0-bailout.test: ... these new tests, and extend
+ a little.
+ * tests/Makefile.am (tap_with_common_setup_tests): Update.
+
+2011-08-17 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap: correctly handle string "0" in TAP messages
+ * lib/tap-driver.pl (is_null_string): New function, can be used
+ to determine whether a given string variable is empty or undefined.
+ Useful to avoid pitfalls like:
+ if ($message) { print "$message\n"; }
+ which wouldn't print anything if $message is the literal "0".
+ (handle_tap_test, handle_tap_plan, handle_tap_bailout): Use it,
+ to avoid missing messages composed only by a literal "0" in TAP
+ result descriptions and in skip, todo and bailout explanations.
+ * tests/tap-message-0.test: Enhance.
+ * tests/Makefile.am (XFAIL_TESTS): Remove it, it passes now.
+
+2011-08-17 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap: a minor simplification in the perl TAP driver
+ * lib/tap-driver.pl: The `--disable-hard-errors' option is a
+ no-op, so just ignore it and its argument.
+
+2011-08-17 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ parallel-tests: fix help screen for test driver scripts
+ * lib/tap-driver.pl ($USAGE): The `--trs-file' option is mandatory
+ too.
+ * lib/tap-driver.sh (print_usage): Likewise.
+ * lib/test-driver (print_usage): Likewise.
+ ($scriptversion): Update.
+
+2011-08-17 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap: add a dummy TAP driver script implemented in shell + awk
+ The user can also now decide which implementation of the TAP driver
+ to use in the testsuite by defining the `$am_tap_implementation'
+ variable to either "perl" or "shell". Future enhancements will
+ allow the testsuite to automatically run the test scripts on TAP
+ support with both the TAP driver implementations, to improve
+ coverage.
+ * tests/defs (fetch_tap_driver): Honor the `$am_tap_implementation'
+ variable to decide which implementation of the TAP driver to fetch.
+ ($am_tap_implementation): Default to "perl".
+ * tests/tap-common-setup.test: Do not fetch the TAP driver, the
+ code in tap-setup.sh does that already (and respecting runtime
+ overriding of `$am_tap_implementation').
+ * lib/tap-driver: Renamed ...
+ * lib/tap-driver.pl: ... to this, and ...
+ ($ME): ... adjusted this.
+ * doc/automake.texi: Adjust to the renaming.
+ * tests/Makefile.am (TAP_LOG_DRIVER): Likewise.
+ * tests/tap-doc2.test: Likewise.
+ * lib/tap-driver.sh: New script, still mostly dummy.
+ * lib/Makefile.am (dist_script_DATA): Update, and since we are at
+ it, rewrite it to make it easier to add new entries in the future.
+
+2011-08-17 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: refactor tests on TAP support in view of future changes
+ * tests/defs (fetch_tap_driver): New subroutine; it fetches the
+ automake-provided TAP driver from the `lib/' directory into the
+ current directory, and edits its shebang line so that it will be
+ run with the perl interpreter determined at configure time.
+ * tests/tap-setup.sh: Use it.
+ * tests/tap-common-setup.test: There's no need to AC_SUBST `PERL'
+ anymore, nor to use it in the Makefile to run the TAP driver.
+ Also, use the `fetch_tap_driver' function instead of copying the
+ `tap-driver' auxiliary script directly.
+ * tests/tap-bad-prog.tap: Likewise.
+ * tests/tap-diagnostic-custom.test: Likewise.
+ * tests/tap-doc.test: Likewise.
+ * tests/tap-merge-stdout-stderr.test: Likewise.
+ * tests/tap-more.test: Likewise.
+ * tests/tap-more2.test: Likewise.
+ * tests/tap-recheck.test: Likewise.
+ * tests/tap-summary-aux.sh: Likewise.
+ * tests/tap-basic.test: Likewise, and fix a grammaro in comments
+ since we are at it.
+
+2011-08-17 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ test defs: fix typo in comments
+ * tests/defs (get_shell_script): Fix typo in the function
+ description.
+
+2011-08-14 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ coverage: missing tap plan and non-zero exit status
+ * tests/tap-missing-plan-and-bad-exit.test: New test.
+ * tests/Makefile.am (tap_with_common_setup_tests): Update.
+
+2011-08-12 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: fix an unportable use of sed
+ * tests/tap-todo-skip-whitespace.test: Alternation with "\|" in
+ sed regular expressions is not portable to (at least) FreeBSD
+ sed; do not use it.
+
+2011-08-12 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: avoid spurious failure if rst2html program is missing
+ * tests/test-driver-custom-no-html.test: Account for the
+ possibility of a lacking `rst2html' program in our final
+ grepping of make output.
+
+2011-08-12 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap: improve diagnosing and reporting of plan mismatches
+ Problem exposed by a failure in the test 'tap-plan-errors.test'.
+ * lib/tap-driver (NO_PLAN, EARLY_PLAN, LATE_PLAN): New constants.
+ (Throughout the file): Use more complex semantics for `$plan_seen',
+ which now also remember whether the plan that has been seen is an
+ "early" or "late" plan; in turn, this makes ...
+ ($tap_stopped): ... this variable redundant; remove it.
+ (handle_tap_plan): Adjust to avoid producing spurious or confusing
+ error messages about extra or mismatched "late" TAP plans.
+ * tests/tap-plan-corned.test: Adjust and extend.
+ * tests/tap-color.test: Likewise.
+
+2011-08-12 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap: log all TAP stream, even after a "Bail out!"
+ * lib/tap-driver ($bailed_out): New global boolean variable,
+ telling whether a "Bail out!" directive has been seen or not.
+ (handle_tap_bailout): This function does not anymore stop the
+ reading from TAP stream; instead, it sets `$bailed_out' to a
+ true value, so that only the subsequent parsing of the input
+ TAP stream is stopped.
+ (finish): Remove, no more needed, its contents inlined into ...
+ (main): ... this function, with related adjustments in the code
+ flow.
+ (get_test_exit_message): Do not "flush" the input TAP stream
+ to fetch the exit status of test script, it is not anymore
+ required. Add a sanity check.
+ * tests/tap-bailout-and-logging.test: New test.
+ * tests/Makefile.am (tap_with_common_setup_tests): Update.
+
+2011-08-12 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ coverage: TAP diagnostics after "Bail out!" aren't reported
+ This is compatible with the behaviour of the `prove' utility.
+ * tests/tap-bailout-suppress-later-diagnostic.test: New test.
+ * tests/Makefile.am (tap_with_common_setup_tests): Update.
+
+2011-08-12 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ coverage: TAP errors after a "Bail out!" directive aren't reported
+ This is compatible with the behaviour of the `prove' utility.
+ * tests/tap-bailout-suppress-later-errors.test: New test.
+ * tests/tap-bailout-badexit.test: Renamed ...
+ * tests/tap-bailout-suppress-badexit.test: ... to this, for
+ consistency.
+ * tests/Makefile.am (tap_with_common_setup_tests): Update.
+
+2011-08-12 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: reorganize tests on TAP plans
+ * tests/tap-skip-whole.test: Rename ...
+ * tests/tap-planskip.test: ... to this
+ * tests/tap-skip-whole-badexit.test: Rename ...
+ * tests/tap-planskip-badexit.test: ... to this.
+ * tests/tap-skip-whole-bailout.test: Rename ...
+ * tests/tap-planskip-bailout.test: ... to this.
+ * tests/tap-skip-whole-bailout.test: Rename ...
+ * tests/tap-planskip-later-errors.test: ... to this.
+ * tests/tap-skip-whole-unplanned.test: Rename ...
+ * tests/tap-planskip-unplanned.test: ... to this.
+ * tests/tap-skip-whole-whitespace.test: Rename ...
+ * tests/tap-planskip-whitespace.test: ... to this.
+ * tests/tap-skip-whole-continue.test: Rename ...
+ * tests/tap-planskip-and-logging.test: ... to this, extend to try
+ more cases, and fix heading comments.
+ * tests/tap-skip-whole-lastline.test: Rename ...
+ * tests/tap-planskip-late.test: ... to this, fix heading comments,
+ and remove some checks that are just a duplication of those already
+ performed in the previous script.
+ * tests/tap-results-with-planskip.test: Rename to ...
+ * tests/tap-planskip-unplanned-corner.test: ... this, and improve
+ hading comments.
+ * tests/tap-planskip-later-errors.test: New test.
+ * tests/Makefile.am (tap_with_common_setup_tests): Update.
+
+2011-08-12 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ coverage: more TAP driver consistency w.r.t. 'prove'
+ * tests/skip-results-with-planskip.test: New test, verifies that
+ test results seen in a TAP stream that has a "plan with SKIP" are
+ flagged as errors.
+ * tests/Makefile.am (tap_with_common_setup_tests): Update.
+
+2011-08-12 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap: improve some comments in the TAP driver
+ * lib/tap-driver (handle_tap_plan): Improve comments describing
+ possible errors in a "plan with SKIP" directive.
+
+2011-08-12 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap: non-zero exit status after "Bail out!" should not be reported
+ This new behaviour is more consistent with that of the `prove'
+ utility. Also, experience with the use of TAP in the Automake's
+ own test suite has shown that reporting non-zero exit status from
+ a script after it has issued a "Bail out!" directive is mostly
+ redundant, more introducing noise rather than conveying useful
+ information.
+ * lib/tap-driver (finish): Inline the part of it that was getting
+ the script exit status ...
+ (main): ... in here.
+ * tests/tap-bailout-badexit.test: New test.
+ * tests/Makefile.am (tap_with_common_setup_tests): Update.
+
+2011-08-11 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap: a "plan with SKIP" after test results is an error
+ This new behaviour is both more natural and more consistent with
+ that of the `prove' utility.
+ * lib/tap-driver (handle_tap_plan): Do not stop TAP parsing when
+ a "plan with SKIP" line (e.g., "1..0 # SKIP") is encountered, and
+ do not print a "SKIP" line if some TAP result has already been
+ seen.
+ * tests/tap-skip-whole.test: Adapt.
+ * tests/tap-skip-whole-lastline.test: Likewise.
+ * tests/tap-global-result.test: Adapt and extend.
+ * tests/tap-skip-plan-errors.test: Likewise, and fix an obsolete
+ small part of a comment.
+ * tests/tap-skip-whole-bailout.test: New test.
+ * tests/tap-skip-whole-unplanned.test: Likewise.
+ * tests/tap-skip-whole-continue.test: Likewise.
+ * tests/Makefile.am (tap_with_common_setup_tests): Update.
+
+2011-08-09 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ test harness: avoid possible fork bomb
+ This fixes a regression w.r.t. the master branch, exposed by
+ test 'parallel-tests-fork-bomb.test'.
+ * lib/am/check.am (am--redo-logs): Detect possible infinite
+ recursion due to a test log in $(TEST_LOGS) being the same
+ as $(TEST_SUITE_LOG).
+
+2011-08-09 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ coverage: enhance test against infinite recursion in test harness
+ * tests/parallel-tests-fork-bomb.test: Enhance and extend a
+ little now, for soon-to-be-implemented more explicit and safer
+ semantics.
+
2011-08-09 Stefano Lattarini <stefano.lattarini@gmail.com>
coverage: possible infinite recursion in the test harness
test defs: yet more environment cleanup
* tests/defs: Also unset the TEST_SUITE_LOG variable.
+2011-08-14 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ self tests: some minor refactoring
+ * tests/self-check-exit.tap: Disable the `errexit' shell flag;
+ this allow to remove some boilerplate idioms and to make some
+ code slightly less cumbersome.
+ * tests/self-check-me.tap: Likewise.
+
+2011-08-14 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ maintcheck: look for TAP-based tests without TAP plan
+ * Makefile.am (sc_tests_tap_plan): New maintainer check.
+ (syntax_check_rules): Add it.
+ * tests/suffix10.tap: Declare a TAP plan.
+ * tests/add-missing.tap: Likewise.
+
+2011-08-14 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: avoid confusing SKIP messages for errored tests
+ * tests/defs [$use_tap] (exit trap): Don't print the test plan
+ in case the client script exited with a non-zero status. This
+ avoids the display of redundant "SKIP" messages for tests with
+ lazy plans that error out before any test is run.
+
+2011-08-13 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: keep more debugging info around in 'add-missing.tap'
+ * tests/add-missing.tap: Use a different directory for each test,
+ so that in case of failure more information is kept around.
+ Since we are at it, remove redundant TAP diagnostic, and define
+ `$am_create_testdir' to "empty" to avoid bringing in unused
+ auxiliary files.
+
+2011-08-13 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: more granular count of test results in our TAP library
+ * tests/tap-functions.sh ($tap_bad_count_): Removed, superseded
+ by ...
+ ($tap_fail_count_, $tap_xpass_count_): ... these new variables,
+ which keep more granular counts.
+ ($tap_pass_count_: New variable.
+ * tests/defs (exit trap): Adjust and simplify accordingly.
+
+2011-08-13 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: fixlets and improvements in two long TAP-based tests
+ * tests/depmod.tap: Clean up the subdirectories used by tests that
+ passed, to avoid ending up with a too big test directory. This is
+ especially important since, on each and every system, at least some
+ tests in this script are expected to be skipped (which ones exactly
+ depends on the system), thus causing the test directory not to be
+ removed when the script terminates.
+ * tests/instspc.tap: Likewise (with the difference that tests here
+ are not expected to be skipped, but to xfail). Give more explicit
+ error message in case of setup failure. Skip the proper number of
+ tests when required, to avoid spurious errors due to mismatched
+ plan. Also, really avoid to run the unsupported test cases when
+ the need to skip them is detected.
+
+2011-08-09 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ test harness: recursive make invocations must use $(AM_MAKEFLAGS)
+ * lib/am/check.am ($(TEST_SUITE_LOG)): Pass $(AM_MAKEFLAGS)
+ to $(MAKE) when issuing a recursive make call.
+ Problem detected by the `sc_tests_make_without_am_makeflags'
+ maintainer check.
+
+2011-08-09 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: fix spurious failure in 'suffix-chain.tap'
+ * tests/suffix-chain.test: Use more uncommon suffixes that
+ don't trigger built-in recipes with FreeBSD make.
+
+2011-08-09 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: improvements to our shell TAP library and APIs
+ These improvements impact only the Automake's own testsuite.
+ * tests/tap-functions.sh (plan_): Allow new special arguments
+ "unknown" a.k.a. "later" and "lazy" a.k.a. "now". Perform
+ sanity check the given arguments. Adjust incorrect comments.
+ (lazy_plan_): Removed, superseded by the new version of `plan_'.
+ (not): New function, run a command and inverts its exit status.
+ (ok_, not_ok_, skip_): These are just shorthands for common
+ `result_' usages.
+ (xfail_, xpass_, reset_test_count_): Removed, they are not used,
+ nor does a legitimate use for them seem in sight.
+ (not): New function, to run a command inverting its exit status.
+ (command_not_ok_): Removed, superseded by `command_ok_' + `not'.
+ (command_ok_): Extended to accept TAP directives.
+ * tests/defs (trap): Don't use `late_plan_' anymore, use
+ `plan_ "now"' instead.
+ * tests/ac-output-old.tap: Update to take advantage of the new
+ APIs.
+ * tests/self-check-env-sanitize.tap: Likewise.
+ * tests/suffix-chain.tap: Likewise.
+ * tests/suffix10.tap: Likewise.
+ * tests/suffix11.tap: Likewise.
+ * tests/suffix3.tap: Likewise.
+ * tests/suffix8.tap: Likewise.
+ * tests/tap-bad-prog.tap: Likewise.
+ * tests/self-check-exit.tap: Likewise, plus a small improvement
+ in the name of a variable.
+
2011-08-09 Stefano Lattarini <stefano.lattarini@gmail.com>
- tests: fix various blunders in 'suffix-chain.test'
- * tests/suffix-chain.test: Fix various blunders that were causing
- this test to fail spuriously: append to `configure.in', not to
- `configure.ac'; add a `;' character after the `return' statement
- in an input file; use tabs, not spaces, to indent Makefile recipes;
- and use more uncommon suffixes that don't trigger built-in recipes
- with FreeBSD make.
+ testsuite: convert some `suffix*' tests to TAP
+ This will allow us to explicitly declare some expected failures
+ involving Solaris make as such, instead of partly papering over
+ them with skips and partly letting them fail too loudly.
+ * tests/suffix-chain.test: Rename ...
+ * tests/suffix-chain.tap: ... to this, and convert to the use of
+ TAP. Also, fix some bugs since we are at it, and ...
+ ($required): ... require `cc'.
+ * tests/suffix3.test: Rename ...
+ * tests/suffix3.tap: ... to this, and convert to the use of TAP.
+ Also, since we are at it, throw in some formatting fixes.
+ * tests/suffix8.test: Rename ...
+ * tests/suffix8.tap: ... to this, and convert to the use of TAP.
+ Also, since we are at it, throw in some formatting fixes.
+ * tests/suffix10.test: Rename ...
+ * tests/suffix10.tap: ... to this, and convert to the use of TAP.
+ * tests/suffix11.test: Rename ...
+ * tests/suffix11.tap: ... to this, and convert to the use of TAP.
+ * tests/Makefile.am (TESTS): Update.
2011-08-09 Stefano Lattarini <stefano.lattarini@gmail.com>
2011-08-09 Stefano Lattarini <stefano.lattarini@gmail.com>
- testsuite: avoid spurious failure of self checks on Solaris
- This is just a quick workaround to prevent having too much noise
- in the testsuite; more proper fixes have been committed into the
- 'testsuite-work' branch.
- * tests/self-check-dir.test: Relax a test to avoid unimportant
- failure on Solaris 10.
- * tests/self-check-exit.test: Likewise.
+ self tests: avoid spurious output on console, and related fixlets
+ * tests/self-check-cleanup.test: Set `$stderr_fileno_' to `2'
+ in the self tests, to avoid spurious diagnostic on console.
+ * tests/self-check-exit.tap: Only unset `am_explicit_skips' and
+ `$stderr_fileno_' in the self tests, not at the global level, it
+ is more correct.
+ ($init): New variable, to reduce code duplication.
+ Some refactoring and adjustments to the name of the tests.
+
+2011-08-09 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ test defs: few more sanity checks
+ * tests/defs: Catch some more (very unlikely) set-up or internal
+ errors, and diagnose them explicitly. This is intended mostly
+ to be a theoretic-only improvement.
+
+2011-08-09 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ test defs: really make them "set -e" clean
+ Motivated by a spurious failure of test `instsh3-w.shtst' on
+ Solaris 10.
+ * tests/defs: Make really "set -e" clean, to avoid spurious
+ failures in case this file is sourced by a test script that has
+ already set the `errexit' flag. To be sure we don't regress,
+ move the `set -e' call much earlier, and make the comment telling
+ about the expected "set -e" cleanliness more loud and outstanding.
+
+2011-08-09 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ self tests: register an expected failures with Solaris /bin/sh
+ Solaris 10 /bin/sh erroneously exit with success right away when
+ the following three conditions are met at the same time:
+ 1. the `errexit' flag is active,
+ 2. an exit trap is installed, and
+ 3. a non-existing command is issued.
+ * tests/self-exit.tap: When that bug is detected, issue an XFAIL
+ rather than a FAIL. Since we are at it, improve by avoiding
+ creation of stray temporary files when testing for non-executable
+ commands.
+
+2011-08-09 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ gitignore: update and improve for the testsuite files
+ * tests/.gitignore: Don't ignore temporary partial log files,
+ autogenerated tests and makefile fragments that are not used
+ anymore. Anchor all the remaining patterns.
+
+2011-08-09 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ self tests: some new uses of TAP, few extensions
+ * tests/self-check-me.test: Renamed ...
+ * tests/self-check-me.tap: .. to this, and converted to use TAP.
+ * tests/self-check-exit.test: Renamed ...
+ * tests/self-check-exit.tap: .. to this, and converted to use TAP.
+ Set `$stderr_fileno_' to `2', to avoid spurious messages on the
+ console. Add a new test, and relax another one.
+ * tests/self-check-env-sanitize.test: Renamed ...
+ * tests/self-check-env-sanitize.tap: .. to this, and converted to
+ the use TAP.
+ * tests/Makefile.am (TESTS): Update.
+
+2011-08-09 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ test defs: test directory creation more configurable
+ After this change, the individual tests will be able to tell
+ `./defs' whether or not to create a temporary test directory
+ for them, and if yes, whether or not to pre-populate it.
+ * tests/defs: Only create the temporary test directory if the
+ variable `$am_create_testdir' is not set to "no". In this
+ case, only pre-populate it if that variable is not set to
+ `$empty'.
+ * tests/defs-static.in: Check that `$am_create_testdir' is not
+ exported in the environment.
+ * tests/self-check-env-sanitize.test: Update.
+ * tests/self-check-dir.test: Extend, and make deeper and more
+ robust.
+ * tests/README (Section "Writing test cases" subsection "Do"):
+ Update.
+ * tests/autodist.test: Set `$am_create_testdir' instead of
+ removing by hand files created by ./defs.
+ * tests/tap-doc2.test: Likewise.
+
+2011-08-08 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: more uses of the `seq_' function
+ * tests/parallel-tests-exit-statuses.test: Use the `seq_' function
+ from `tests/defs', instead of duplicating its semantics.
+ * tests/testsuite-summary-count.test: Likewise.
+
+2011-08-08 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: drop redundant environment sanitization
+ * tests/Makefile.am (TESTS_ENVIRONMENT): Don't remove `planned'
+ from the environment.
+ * tests/defs-static.in: Don't complain if `planned' is set in
+ the environment.
+ * tests/self-check-env-sanitize.test: Update.
+
+2011-08-08 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ test defs: fix bug in plain version of `skip_all_'
+ * tests/plain-functions.sh (skip_all_): Call `skip_', not `skip'.
+
+2011-08-08 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ test defs: small cleanups and tweakings
+ * tests/defs: Where possible, prefer `framework_failure_' or
+ `fatal_' to `Exit 99'. Try to catch more possible hard errors
+ and setup failures. Do not hide expected error messages. Avoid
+ a couple of forks when copying files from `$(top_srcdir)/lib'.
+ ($signal): Remove, no more needed.
+
+2011-08-08 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: improve interface for giving early TAP plan
+ Since commit `v1.11-976-g4740f56', an early TAP plan can also be
+ specified after input lines have already been seen, as long as
+ no one of them is a TAP result line. We can now take advantage
+ of this in our own testsuite, allowing the TAP-based scripts to
+ specify their earlier plan after having included `./defs'. In
+ particular, this offers the benefit of allowing those scripts to
+ issue a "skip all" plan (e.g., "1..0 # SKIP reason ...") after
+ the inclusion of `./defs', which wasn't possible with the older
+ interface (introduced in commit `v1.11-1114-g0286308').
+ * tests/defs: Don't try to call `plan_' if `$planned' is defined,
+ the calling scripts can do that directly now.
+ * tests/ac-output-old.tap: Use `plan_' directly instead of
+ setting `$planned' before including `./defs'.
+ * tests/automake-cmdline.tap: Likewise.
+ * tests/depmod.tap: Likewise.
+ * tests/instspc.tap: Likewise.
+ * tests/tap-bad-prog.tap: Likewise.
+ * tests/add-missing.tap: Don't set `$planned'.
+
+2011-08-08 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ test defs: fix bug in TAP version of `skip_all_'
+ * tests/tap-functions (skip_all_): Set `$have_tap_plan_' to "yes"
+ when called, so that the exit trap won't write another TAP plan.
+
+2011-08-08 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ test defs: fix skip directives to work also with TAP-based tests
+ * test/defs: Use `skip_all_', not `skip_', to issue a skip of the
+ whole calling test script.
2011-08-08 Stefano Lattarini <stefano.lattarini@gmail.com>
2011-08-08 Stefano Lattarini <stefano.lattarini@gmail.com>
+ test harness: more minor VPATH-rewrite issues
+ * tests/am/check.am (am__set_TESTS_bases, $(TEST_SUITE_LOGS),
+ am--redo-logs, recheck, recheck-html): Cosmetic fixlets to
+ minimize the risk of unwanted VPATH rewrites.
+ (check-TESTS): Likewise, and normalize trailing whitespace
+ since we are at it.
+ Bugs exposed by test cases `check6-p.test' and `check7-p.test'.
+
+2011-08-08 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ test harness: "make -n check" works correctly with Solaris make
+ To avoid some recipes being executed with "make -n", we used the
+ trick of issuing recursive make calls not directly with $(MAKE),
+ but with a simple indirection instead:
+ ...
+ am__stealth_MAKE = $(MAKE)
+ ...
+ target:
+ ... $(am__stealth_MAKE) ...
+ ...
+ This hack worked with GNU make, but, unfortunately, not with
+ Solaris make. The present change removes the hack, and uses more
+ proper idioms to avoid unwanted recipe executions with "make -n".
+ Bug exposed by test case `parallel-tests-dry-run.test'.
+ * lib/am/check.am (am__stealth_MAKE): Remove, no more used.
+ (.log.trs): Break recipe in multiple shell invocations.
+ ($(TEST_SUITE_LOG)): Likewise, and move some corner-case parts of
+ the recipe to ...
+ (am--redo-logs): ... this new internal helper .PHONY target.
+
+2011-08-08 Stefano Lattarini <stefano.lattarini@gmail.com>
+
tests: fix spurious failure with Solaris make
* tests/distcheck-configure-flags-am.test: Avoid using `+=' too
liberally with AM_DISTCHECK_CONFIGURE_FLAGS, since the line breaks
so introduced, in conjunction with single quotes, might confuse
Solaris make.
+2011-08-08 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ test harness: fixlet in workaround for VPATH rewrite issue
+ Fixes a minor in previous commit `v1.11-983-gfda3de5'.
+ * lib/am/check.am (am__set_TESTS_bases): Add forgotten end-of-line
+ anchor to sed expression. In comments, add reference to ...
+ * tests/test-trs-basic.test: ... this test, which has been adapted
+ and tightened (and tweaked).
+
+2011-08-08 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ test harness: work around a VPATH rewrite issue
+ * lib/am/check.am (am__TEST_BASES): Removed, it's role taken
+ over by ...
+ (am__set_TESTS_bases): ... these new variable.
+ ($(TEST_SUITE_LOG): Use it, to avoid VPATH rewrite issues.
+ * automake.in (handle_tests): Update the code for the cleanup
+ of the `.trs' file to use `$(TEST_LOGS)' instead of
+ `$(am__TEST_BASES)'.
+ * tests/test-harness-vpath-rewrite.test: New test, better
+ exposing the problem fixed by this change
+ * tests/Makefile.am (TESTS): Add it.
+ * tests/test-trs-basic.test: Update and extend.
+
+2011-08-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ parallel-tests: work around Solaris XPG4 make segfault
+ Issue exposed by test `posixsubst-tests-p.test', and similar to
+ the problem solved by commit `v1.11-159-ge7aa360'.
+ * lib/am/check.am [%?PARALLEL_TESTS%] (check-TESTS): Trim trailing
+ whitespace from $list, to avoid triggering a nasty bug (potential
+ segfault) on Solaris XPG4 make and Heirloom make.
+
+2011-08-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: fix weird spurious failure with Solaris /bin/sh
+ Solaris /bin/sh, when killed with a SIGTERM or SIGINT signal, can
+ apparently end up exiting with exit status 208, instead of leaving
+ the correct wide exit status to the parent. See:
+ <http://dbaspot.com/shell/396118-bourne-shell-exit-code-term.html>
+ Work around this incompatibility.
+ * tap-signal.test: Look for the above Solaris bug.
+ (signal_caught): Adapt to handle it.
+
+2011-08-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: fix another spurious failure on Solaris make
+ * tests/parallel-tests-log-override-recheck.test: Filter make
+ output before grepping it, for make implementations that, like
+ Solaris' one, print the whole of the failed recipe on failure.
+
+2011-08-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: fix two spurious failures on Solaris make
+ * tests/tests-environment-and-log-compiler.test: Do not fail if
+ "make -n check" fails, as that is issued only for debugging, its
+ semantics being tested in other test scripts.
+ * tests/test-driver-strip-vpath.test: Likewise.
+
+2011-08-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap driver: refactor and remove dead code
+ * lib/tap-driver (get_tap_line): Removed, ...
+ (all callers): ... updated to use $parser->next directly instead.
+ (peek_tap_line, unget_tap_line, @tap_lines): Remove, they're not
+ used anymore.
+ (TAP_PEEKING): Block deleted, all its subroutines and variables
+ having been removed.
+
+2011-08-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: fix spurious failure in a test on TAP support
+ * tests/tap-passthrough-exit.test: When the `--ignore-exit' flag
+ of the TAP driver is used, don't look for a message reporting the
+ non-zero exit statuses of tests in the log files; such message is
+ not expected to be there anymore. Related simplifications.
+
+2011-08-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap: plan location is more liberal w.r.t. non-TAP lines
+ With this change, only lines that are TAP results will matter
+ w.r.t. the position of the TAP plan in the input; for example,
+ this input:
+ this is a non-TAP line
+ # and this a TAP diagnostic line
+ 1..1
+ ok 1
+ was considered to be an error, diagnosed with a message "test
+ plan in middle of output"; as effect of the current change, such
+ input is now valid. This is more consistent with the behaviour
+ of the `prove' utility.
+ * lib/tap-driver ($lineno): Removed, no more needed.
+ ($tap_stopped): New global variable.
+ (stringify_test_result): Return "ERROR" if a TAP result is found
+ when `$tap_stopped' is set to true.
+ (handle_tap_test): Diagnose TAP results that comes after a late
+ plan. Add a couple of blank lines, for clarity.
+ (handle_tap_plan): Set `$tap_stopped' to true after a late plan
+ is encountered. Do not complain anymore for extra non-TAP lines
+ preceding or following the plan. Adjust comments.
+ (main): Don't increment $lineno anymore.
+ * tests/tap-plan.test: Extend a bit, and remove stale comment.
+ * tests/tap-color.test: Adjust.
+ * tests/tap-passthrough.test: Likewise.
+ * tests/tap-plan-corner.test: Adjust and extend.
+ * tests/tap-plan-errors.test: Likewise.
+ * tests/tap-plan-middle.test: New test.
+ * tests/tap-plan-corner2.test: Delete, it's obsolete now.
+ * tests/Makefile.am (XFAIL_TESTS): Remove it.
+ (tap_with_common_setup_tests): Likewise, and add
+ `tap-plan-corner.test'.
+
+2011-08-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: remove now-passing test from XFAIL_TESTS
+ * tests/Makefile.am (XFAIL_TESTS): Remove `tap-signal.test',
+ which is passing since previous commit `v1.11-974-gc7fa872'.
+
+2011-08-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ test defs: get name of TAP-using tests correctly
+ This change fixes the names of the temporary directories used
+ by the TAP-based tests (e.g., test `foo.tap' now uses `foo.dir'
+ rather than `foo.tap.dir'); more importantly, this make the
+ variable `$destdir' defined in `tests/defs' usable also in the
+ TAP tests.
+ ($me): Also strip the any `.tap' suffix from the name of the
+ test.
+
+2011-08-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: TAP tests properly decide when to remove tempdirs
+ Before this change, the TAP tests in the Automake testsuite were
+ removing the temporary test directory even when they failed or
+ were skipped, thus making debugging more difficult.
+ * tests/tap-functions.sh (incr_tap_count): Removed, superseded
+ by ...
+ (incr_): ... this function, which can increment the value of any
+ variable passed to it.
+ (result_): Updated to use `incr_' instead of the now-removed
+ `incr_tap_count_'. Keep count of failures, xfailures, xpasses,
+ and skips, using ...
+ ($tap_skip_count_, $tap_bad_count, _$tap_xfail_count_): ...
+ these new variables.
+ * tests/defs (trap): Try to use their values to decide whether
+ the temporary directory being used by the test script should be
+ removed or not. Other code reorganizations. And move the code
+ for the removal of the temporary directory out to ...
+ (rm_rf_): ... this new subroutine.
+ (Main code): Use that instead of duplicating the code.
+
+2011-08-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: improve and refactor our custom TAP shell library
+ This change is mostly done in preparation for the next one.
+ * tests/tap-functions.sh (result_with_directive_): Merge into ...
+ (result_): ... this function, which has been greatly extended and
+ improved in various ways.
+ (ok_, not_ok_, skip_, xfail_, xpass_): Adjust.
+ Adjust comments. Remove an extra empty line.
+ * tests/instspc.tap: Use the new improved interface of `result_'.
+ * tests/tap-bad-prog.tap: Likewise.
+
+2011-08-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: update "user interface" description in tests/README
+ * tests/README (Subsection "Interpretation"): Mention the "ERROR"
+ outcome.
+ (Subsection "Getting details from failures"): Don't tell that the
+ test scripts can be run directly, this is not 100% true; anymore;
+ instead ...
+ (Subsection "About the tests"): ... use this new subsection to
+ explain how to run the tests (either simple or TAP-based) by hand.
+ (Subsection "Supported shells"): Extend to refer to TAP tests too.
+
+2011-08-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: TAP tests can have early plan now
+ * tests/defs: When using TAP, allow the user to specify the number
+ of tests through the `$planned' variable.
+ (trap): Do not print the auto-generated plan if an explicit one
+ has already been printed.
+ * tests/defs-static.in: Complain and bail out if the `$planned'
+ variable is exported in the environment. Reformat the code to
+ facilitate future additions.
+ * tests/Makefile.am (TESTS_ENVIRONMENT): Neutralize `$planned'.
+ * tests/self-check-env-sanitize.test: Update.
+ * tests/ac-output-old.tap: Give an explicit TAP plan, by defining
+ `$planned' just before including `./defs'.
+ * tests/depmode.tap: Likewise.
+ * tests/automake-cmdline.tap: Likewise.
+ * tests/tap-bad-prog.tap: Likewise.
+ * tests/instspc.tap: Likewise, and related adjustments.
+ * tests/add-missing.tap: Make it explicit that we have no a-priori
+ plan, and tell why.
+
+2011-08-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: run autogenerated tests with $(LOG_COMPILER) too
+ This change enhances consistency in the testsuite.
+ * tests/Makefile.am: Some cosmetic reordering, to keep the
+ definition of FOO_LOG_COMPILER near to the place where `.foo'
+ is added to $(TEST_EXTENSIONS).
+ (PTEST_LOG_COMPILER): Run `$(srcdir)/parallel-tests.sh' with
+ `$(LOG_COMPILER)', not with `$(SHELL)'.
+ (SHTST_LOG_COMPILER): Likewise.
+
+2011-08-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: run TAP and "plain" tests with the same $(LOG_COMPILER)
+ With this change, it will be easier to override, at make time, the
+ shell used to run the tests.
+ * tests/Makefile.am (LOG_COMPILER): Define to `$(SHELL)'.
+ (TEST_LOG_COMPILER): Re-define to `$(LOG_COMPILER)'.
+ (TAP_LOG_COMPILER): Likewise.
+
+2011-08-06 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: run TAP-based tests explicitly with $(SHELL)
+ * tests/Makefile.am (TAP_LOG_COMPILER): Define to `$(SHELL)'.
+
+2011-08-06 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap driver: handle signals received by the tests being run
+ * lib/tap-driver (get_test_exit_message): Also deal with signals,
+ by using the `wait' method of the TAP::Parser object instead of
+ the `exit' method. This required the use of the standard perl
+ module `POSIX'.
+ * doc/automake.texi (Use TAP with the Automake test harness):
+ Document that `--ignore-exit' has effect also on terminating
+ signals. Add a "synchronizing" comment that references the tests
+ 'tap-exit.test' and 'tap-signal.test'.
+ * tests/tap-signal.test: Extend and adjust.
+
+2011-08-06 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ test driver: a preparatory refactoring (2)
+ * lib/tap-driver (finish): Move code to fetch the message about
+ the exit status of the test being run to ...
+ (get_test_exit_message): ... this new subroutine.
+
+2011-08-06 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap driver: a preparatory refactoring (1)
+ * lib/tap-driver (finish): Do not flush the TAP stream nor
+ fetch the exit status of the TAP producer if running with
+ the `--ignore-exit' option. Obviously, don't try to write
+ such exit status in the test log anymore.
+
+2011-08-06 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ cosmetics: typofix in the name of a function in the TAP driver
+ * lib/tap-driver (testuite_error): Renamed ...
+ (testsuite_error): ... to this, and minor related reformatting.
+
+2011-08-06 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap: scripts with a SKIP plan but with exit status != 0 must error
+ This change has been motivated by Automake's own testsuite. Some
+ TAP tests there were erroring out (due to unexpected, unhandled
+ failures) before having encountered TAP result, so that the
+ simple-minded shell library implementing TAP generation ended up
+ printing a "1..0" trailing test plan; this caused the script to be
+ reported as a SKIP rather than an ERROR -- a nasty false negative.
+ * lib/tap-driver: Add prototypes for each subroutine, to free up
+ the order in which they can be defined and called.
+ (main): Move the code checking for a bad exit status of the TAP
+ producer ...
+ (finish): ... here, and flush the TAP stream to ensure that the
+ parser always obtains the producer's exit status.
+ * tests/tap-skip-whole-badexit.test: New test.
+ * tests/Makefile.am (tap_with_common_setup_tests): Add it.
+
+2011-08-06 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap: fix whitespace munging of diagnostic messages
+ * lib/tap-driver (extract_tap_comment): Pass the `g' flag to the
+ substitution operator, to strip also trailing whitespaces. Fixes
+ a failure in test `tap-whitespace-normalization.test'.
+
+2011-08-06 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: refactor and cleanup 'instspc.tap'
+ * tests/instspc.tap (is_in_list): New helper subroutine.
+ (expected_to_fail): Re-implement using it.
+ (define_problematic_string): Likewise. Also, rename the special
+ arguments `build-fail' and 'install-fail' to respectively
+ `builddir-fail' and `destdir-fail', and other related changes.
+ (Test data definition): Adapt.
+ ($instspc_xfail_builds_list): Renamed ...
+ ($builddir_xfails): ... to this.
+ ($instspc_xfail_installs_list): Renamed ...
+ ($destdir_xfails): ... to this.
+ ($instspc_names_list): Renamed ...
+ ($test_names_list): ... to this.
+ ($instspc_test_string): Renamed ...
+ ($test_string): ... to this.
+ Add some explicative and "FIXME" comments.
+
+2011-08-06 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: use TAP for `depmod*' tests, related simplifications
+ * tests/depmod-tests.sh: Delete this complex and multifarious
+ script, moving all it checks it used to perform into ...
+ * tests/depmod.tap: ... this TAP-based test script. And extend
+ it quite a bit, since the new form of the test was papering over
+ some pre-existing failures (this was due to the fact that the
+ values used for the VPATH were shorter in the new test script).
+ * tests/depmod-data.test: Deleted.
+ * tests/Makefile.am ($(srcdir)/depmod-tests.am): Don't include
+ nor generate it anymore.
+ (EXTRA_DIST): Don't distribute depmod-tests.sh anymore.
+ (TEST_EXTENSIONS): Remove '.depmod'.
+ (DEPMOD_LOG_COMPILER): Deleted.
+ (depmod-data.log, $(depmod_tests:.depmod=.log)): Delete
+ dependencies for these files, which are not used anymore.
+ ($(depmod_tests)): Delete this dummy dependency declaration.
+ (TESTS): Remove $(depmod_tests).
+ (handwritten_tests): Remove depmod-data.test, add depmod.tap.
+ * bootstrap: Don't generate tests/depmod-tests.am anymore.
+
+2011-08-06 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: use TAP for `instspc*' tests, related simplifications
+ * tests/instspc-tests.sh: Delete this complex and multifarious
+ script, moving all it checks it used to perform into ...
+ * tests/instspc.tap: ... this TAP-based test script.
+ * tests/instspc-data.test: Deleted.
+ * tests/Makefile.am ($(srcdir)/instspc-tests.am): Don't include
+ nor generate it anymore.
+ (EXTRA_DIST): Don't distribute instspc-tests.sh anymore.
+ (TEST_EXTENSIONS): Remove '.instspc'.
+ (INSTSPC_LOG_COMPILER): Deleted.
+ (XFAIL_TESTS): Remove $(instspc_xfail_tests).
+ (instspc-data.log, $(instspc_tests:.instspc=.log)): Delete
+ dependencies for these files, which are not used anymore.
+ ($(instspc_tests)): Delete this dummy dependency declaration.
+ (TESTS): Remove $(instspc_tests).
+ (handwritten_tests): Remove instspc-data.test, add instspc.tap.
+ * bootstrap: Don't generate tests/instspc-tests.am anymore.
+
+2011-08-06 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: do fd redirections with $(AM_TESTS_FD_REDIRECT)
+ * tests/Makefile.am: The redirections of file descriptors needed
+ by our test scripts is now done using the $(AM_TESTS_FD_REDIRECT)
+ variable (the new blessed way since commit `v1.11-906-gb9e9d54'),
+ not using an hack involving $(AM_TESTS_ENVIRONMENT). This has
+ the further benefit of allowing the use of $(TESTS_ENVIRONMENT)
+ again on part of the users.
+ * tests/Makefile.am (AM_TESTS_ENVIRONMENT): Remove redirection of
+ file descriptors.
+ (AM_TESTS_FD_REDIRECT): Redirect file descriptor 9 to original
+ stderr.
+ Comments adjusted.
+ * tests/plain-functions.sh ($stderr_fileno_): Update comment.
+
+2011-08-05 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: use custom TAP diagnostic in our own tests
+ This change allow us to easily and transparently avoid spurious
+ TAP diagnostic in our own testsuite, with the help of the new
+ `--diagnostic-string' option of our TAP test driver.
+ * tests/Makefile.am (TAP_LOG_DRIVER_FLAGS): Also pass option
+ `--diagnostic-string' with value `#%#' (necessarily obtained
+ through some shell indirections).
+ * tests/tap-functions.sh (diag_): Assume that the string denoting
+ TAP diagnostic is stored in the `$diag_string_' variable. Update
+ comments.
+ ($diag_string_): Initialize to "#".
+ * tests/defs ($diag_string_): Re-define to "#%#".
+ (show_): Remove, it's obsolete now.
+ * add-missing.tap: Don't use `show_' anymore to avoid spurious
+ TAP diagnostic; `cat' should suffice now.
+
+2011-08-05 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap: fix typo in TAP driver version message
+ * lib/tap-driver (anonymous subroutine printing the script version
+ message): Add missing trailing newline to the version message.
+
+2011-08-05 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap docs: we don't support pragma or version directives (yet)
+ * doc/automake.texi (Incompatibilities with other TAP parsers
+ and drivers): Document that limitation. Normalize an unrelated
+ "TODO" item.
+
+2011-08-05 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap: new option to change the string designating TAP diagnostic
+ Here we introduce a new option `--diagnostic-string' in our TAP
+ test driver, that allows the user to specify which string should
+ denote the beginning of a TAP diagnostic line. This change is
+ not gratuitous, nor result if over-engineering: it is motivated
+ by real issues that have emerged during the use of TAP in the
+ Automake's own testsuite (see the commit `v1.11-1082-g9b967c2'
+ "testsuite: yet more use of TAP, and related extensions").
+ * doc/automake.texi (Use TAP with Automake test harness): Document
+ the new option.
+ (Incompatibilities with other TAP parsers and drivers): Report it
+ as a potential source of incompatibility.
+ * lib/tap-driver ($diag_string): New global variable, defaulting
+ to "#", and whose value can be changed ...
+ (Getopt::Long::GetOptions): ... by the newly recognized option
+ `--diagnostic-string'.
+ (handle_tap_comment): Subroutine removed, some of its simple logic
+ inlined ...
+ (main): ... in here, where now ...
+ (extract_tap_comment): ... this new subroutine is used.
+ ($USAGE): Adjust.
+ * tests/tap-diagnostic.test: Make one check slightly stricter.
+ * tests/tap-diag-custom.test: New test.
+ * tests/Makefile.am (tap_other_tests): Add it.
+
+2011-08-05 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ check: add small "synchronization" comment to `test-driver'
+ * lib/test-driver: Add comment to the code initializing ANSI color
+ escapes, telling to keep it in sync with the similar initialization
+ in `lib/am/check.am'.
+
+2011-08-05 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ maintcheck: fix spurious failure w.r.t. use of bare "exit"
+ * tests/check12.test: Cosmetic changes to avoid triggering a
+ spurious failure of the `sc_tests_Exit_not_exit' maintainer
+ check.
+
+2011-08-05 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ check: update comments to reflect recent heavy changes
+ * lib/am/check.am [%?PARALLEL_TESTS%]: Update the comments to
+ reflect the recent changes. Remove quite many comments that
+ were merely duplicating excerpts from the Automake manual.
+
+2011-08-05 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ check: small cleanups and refactorings in test harness and drivers
+ * lib/tap-driver (yn): New subroutine, converts a boolean value to
+ a "yes" or "no" string.
+ (write_test_results): Use it, and related reformatting.
+ * lib/test-driver (trap): Also remove the `.trs' file on signals,
+ for extra safety.
+ (fatal): Remove this function, it's never used.
+ ($logfile, $trsfile): Renamed ...
+ ($log_file, $trs_file): ... to these, for clarity, and in order to
+ be more consistent with the `tap-driver' script.
+ Improve a couple of comments.
+ * automake.in (handle_tests): Don't define anymore the now-obsolete
+ make macro `$(TEST_LOGS_TMP)', nor add it to the list of files to
+ be removed upon "make mostlyclean".
+ * lib/am/check.am ($(TEST_SUITE_LOG)): New shell function `f_ok',
+ tells whether a path refers to an existing, regular, readable file.
+ Use it throughout.
+ (recheck): Be safer w.r.t. make implementation that run recipes
+ with `errexit' shell flag active.
+
+2011-08-05 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: use AM_TESTS_FD_REDIRECT where appropriate
+ * parallel-tests-interrupt.test: Use AM_TESTS_FD_REDIRECT, not
+ AM_TESTS_ENVIRONMENT, to define file descriptors' redirection
+ for "make check". Fix botched comment.
+
+2011-08-05 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ docs: some fixlets in documentation on testsuites support
+ * doc/automake.texi (Script-based Testsuites): Rename node ...
+ (Scripts-based Testsuites): ... to this. Break overly long lines
+ in an example (were causing problems with PDF output).
+ (Parallel Test Harness): Don't intend paragraphs that come after
+ examples or bulleted list.
+ (Overview of Custom Test Drivers Support): Fix typo s/the the/the/.
+ (Declaring Custom Test Drivers): When giving an aside, prefer comma
+ to parentheses, as it disrupts the reading flow less.
+ (API for Custom Test Drivers): Remove a sentence that was basically
+ duplicated from the previous subsection.
+ (Command-line arguments for test drivers): Don't intend paragraph
+ coming after an itemized list. Clarify wording and fix grammaros.
+ Add a couple of cross-references to earlier explanations of `.log'
+ and `.trs' files.
+ (Links and external resources): Node renamed ...
+ (Links and external resources on TAP): ... to this, for clarity.
+ Slightly extend and clarify introductory sentence.
+
+2011-08-04 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ test harness: be aware of more metadata, simplify test drivers
+ * lib/am/check.am ($(TEST_SUITE_LOG)): When producing the
+ global test log, take into account the new metadata field
+ `:global-test-result:, to write nicely formatted RST section
+ titles, instead of leaving this chore to the individual test
+ drivers.
+ (am_rst_section): Re-introduce this variable, as removed in
+ commit `v1.11-870-ga27c9c4'.
+ * lib/test-driver, lib/tap-driver: Write the global test result
+ as metadata in the `.trs' file, not as (part of) a formatted RST
+ subsection title in the `.log' file. Related simplifications;
+ in particular, get rid of temporary files usage.
+ * doc/automake.texi (Log files generation and test results
+ recording): Document the new metadata.
+ * tests/check12.test: Update.
+ * tests/parallel-tests-harderror.test: Likewise.
+ * tests/parallel-tests-interrupt.test: Likewise.
+ * tests/parallel-tests-log-override-2.test: Likewise.
+ * tests/parallel-tests-log-override-recheck.test: Likewise.
+ * tests/parallel-tests-unreadable.test: Likewise.
+ * tests/tap-global-result.test: Likewise.
+ * tests/test-metadata-results.test: Likewise.
+ * tests/test-log.test: Likewise, and another minor unrelated fix.
+ * tests/test-metadata-global-result.test: New test.
+ * tests/Makefile.am (TESTS): Update.
+
+2011-08-03 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap: check that also a trailing TAP plan can hold a skip directive
+ * tests/tap-skip-whole-lastline.test: New test.
+ * tests/Makefile.am (tap_with_common_setup_tests): Add it.
+
+2011-08-04 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: more correct names for a couple of tests
+ * tests/parallel-tests-ext-driver.test: Rename ...
+ * tests/parallel-tests-log-compiler-1.test: ... to this, and
+ adjust heading comments.
+ * tests/parallel-tests-ext-driver-prog.test: Rename ...
+ * tests/parallel-tests-log-compiler-2.test: ... to this, and
+ adjust heading comments.
+ * tests/Makefile.am (TESTS): Update.
+
+2011-08-04 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: fix another spurious failure
+ * tests/parallel-tests-ext-driver.test: Name out custom/dummy
+ log compilers as `foo-compiler' rather `foo-driver'; not only
+ this is less confusing, it also avoids errors due to the dummy
+ log compiler `test-driver' overriding the `test-driver' helper
+ script installed by Automake.
+ * tests/parallel-tests-ext-driver-prog.test: Adjust heading
+ comments.
+
+2011-08-04 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: fix a spurious failure
+ * tests/parallel-tests-ext-driver.test: Exporting of environment
+ variables directly from $(LOG_COMPILER) variables is not allowed
+ anymore. Adjust to this.
+
+2011-08-04 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: yet more use of TAP, and related extensions
+ * tests/defs (show_): New function, display the contents of one or
+ more files on stdout, with bells & whistles (both for cosmetic and
+ practical reasons, the latter aimed at avoiding spurious TAP
+ diagnostic).
+ (AUTOMAKE_run): Extend and adapt to make it compatible with TAP
+ based tests. Since we are at it, make its implementation more
+ namespace-safe, and improve its command-line interface.
+ (AUTOMAKE_fails): Adapt to the new command-line interface of
+ `AUTOMAKE_run'.
+ * tests/add-missing.test: Renamed ...
+ * tests/add-missing.tap: ... to this, and converted to the use
+ of the TAP protocol, and of the new `show_' function.
+ * tests/automake.test: Renamed ...
+ * tests/automake-cmdline.tap: ... to this, and converted to the
+ use of the TAP protocol.
+ * tests/getopt.test: Adapt to the new command-line interface
+ of `AUTOMAKE_run'.
+ * tests/cond44.test: Likewise.
+ * tests/cond45.test: Likewise.
+ * tests/configure.test: Likewise.
+ * tests/license2.test: Likewise.
+ * tests/parallel-am.test: Likewise.
+ * tests/parallel-am3.test: Likewise.
+ * tests/tags2.test: Likewise.
+ * tests/werror3.test: Likewise.
+ * tests/werror4.test: Likewise.
+ * tests/amopts-variable-expansion.test: Likewise.
+ * tests/warnings-win-over-strictness.test: Likewise.
+ * tests/Makefile.am (TESTS): Update.
+ (TAP_LOG_DRIVER_FLAGS): Add `--merge', so that TAP diagnostic is
+ reported in the testsuite progress output.
+
+2011-08-03 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ maintcheck: fix more maintainer-check failures
+ * tests/test-driver-custom-no-html.test (no-rst): Use `EOF',
+ not `EoL', as the here-document delimiter.
+ * tests/test-trs-basic.test: Use `cat + here-doc' rather
+ than `echo' when creating the dummy test scripts, to please
+ maintainer-check.
+ * tests/test-trs-recover.test: Use creative quoting where
+ needed, to please maintainer-check.
+ * tests/parallel-tests-no-color-in-log.test: Likewise.
+ * tests/parallel-tests-dry-run.test: Likewise.
+
2011-08-03 Stefano Lattarini <stefano.lattarini@gmail.com>
maintcheck: fix maintainer-check failures, both real and spurious
the command line.
* tests/Makefile.am (TESTS): Update.
-2011-06-29 Stefano Lattarini <stefano.lattarini@gmail.com>
+2011-08-03 Stefano Lattarini <stefano.lattarini@gmail.com>
- docs: explain why AM_TESTS_ENVIRONMENT must be semicolon-terminated
- * doc/automake.texi (Simple Tests using parallel-tests): Ditto, and
- related adjustments.
- Suggestion by Ralf Wildenhues.
+ gitignore: ignore `*.trs' files in lib/Automake/tests
+ * lib/Automake/tests/.gitignore: Add `*.trs' pattern.
-2011-06-29 Stefano Lattarini <stefano.lattarini@gmail.com>
+2011-08-03 Stefano Lattarini <stefano.lattarini@gmail.com>
- docs: fix unportable example of AM_TESTS_ENVIRONMENT usage
- * doc/automake.texi (Simple Tests using parallel-tests): The
- old example on AM_TESTS_ENVIRONMENT relied on unportable shell
- features, and in particular didn't work with various Korn
- Shells (see also commit `v1.11-925-g29ca903'). Give another
- example, simpler this time, but still inspired to real-world
- usage (the GNU coreutils testsuite).
+ testsuite: list another "forgotten" test script in Makefile.am
+ * tests/Makefile.am (TESTS): Also list test script
+ `test-driver-trs-suffix-registered.test'; it had been introduced
+ in commit `v1.11-910-g0c81b43', but by mistake it wasn't added
+ to the list of tests in the Makefile back then.
-2011-06-23 Stefano Lattarini <stefano.lattarini@gmail.com>
+2011-08-03 Stefano Lattarini <stefano.lattarini@gmail.com>
- docs: avoid a footnote, some related rewordings and improvements
- * doc/automake.texi (Dist): Reword the part about automatically
- distributed files to avoid a footnote. Since we are at it, extend
- a bit, and add an example and a reference to a relevant test case.
+ testsuite: list "forgotten" test script in Makefile.am
+ * tests/Makefile.am (TESTS): Also list the test script
+ `parallel-tests-no-color-in-log.test'; it had been introduced in
+ commit `v1.11-899-geaac33f', but by mistake it wasn't added to
+ the list of tests in the Makefile back then.
-2011-06-23 Stefano Lattarini <stefano.lattarini@gmail.com>
+2011-08-03 Stefano Lattarini <stefano.lattarini@gmail.com>
- docs: minor cosmetic fixes
- * doc/automake.texi: Break few overly long lines, throughout the
- file.
- ("Simple Tests"): Move @vindex for XFAIL_TESTS to the correct
- position, i.e., before and not after the paragraph where it is
- introduced.
- ("Options" @item ansi2knr): Use @pxref instead of @xref. This
- fixes a texinfo warning.
- ("Other things Automake recognizes" @item AM_C_PROTOTYPES): Use
- @pxref instead of @ref.
+ testsuite: remove duplicated mention of a test in Makefile.am
+ * tests/Makefile.am (tap_other_tests): Remove `tap-empty.test',
+ it is already listed in `$(tap_with_common_setup_tests)'.
-2011-06-23 Stefano Lattarini <stefano.lattarini@gmail.com>
+2011-08-03 Stefano Lattarini <stefano.lattarini@gmail.com>
- help: improve text about automatically-distributed files
- This change fixes automake bug#7819.
- * automake.in (usage): Distinguish between files that are always
- automatically distributed when found, and those which are only
- "under certain conditions".
- * doc/automake.texi (Basics of Distribution): Update accordingly.
- * tests/autodist-subdir.test: Update.
- * tests/autodist-no-duplicate.test: Likewise.
- * tests/autodist.test: Likewise.
- (configure.in): Remove useless call to AM_MAINTAINER_MODE.
+ testsuite: one more use of TAP in our own tests
+ * tests/tap-bad-prog.test, tests/tap-bad-prog2.test: Merged
+ into ...
+ * tests/tap-bad-prog.tap: ... this TAP-generating test.
+ * tests/Makefile.am (tap_other_tests): Remove plan-bad-prog.test
+ and plan-bad-prog2.test.
+ (XFAIL_TESTS): Remove plan-bad-prog2.test.
+ (plan-bad-prog2.log): Remove.
-2011-06-23 Stefano Lattarini <stefano.lattarini@gmail.com>
+2011-08-03 Stefano Lattarini <stefano.lattarini@gmail.com>
- refactor: split 'usage' subroutine in automake
- This change is related to automake bug#7819.
- * automake.in (print_autodist_files): New subroutine,
- extracted from ...
- (usage): ... this, which now uses it.
- * tests/autodist-no-duplicate.test: New test.
- * tests/Makefile.am (TESTS): Update.
+ testsuite: scaffolding to allow use of TAP in our own tests
+ Now that Automake has initial support for the TAP test protocol,
+ we can start "eating our own dog food" and rewrite some tests to
+ use TAP; this should allow for better granularity, especially in
+ the control of skips and expected failures. With this change, we
+ set up the initial scaffolding required by the planned TAP tests,
+ and convert few older tests to use TAP, mostly in order to verify
+ that there are no obvious errors. The tests in our testsuite that
+ use the TAP protocol will be marked by the new `.tap' extension.
+ * tests/plain-functions.sh: New file containing definition of
+ helper shell functions used by the "protocol-less" tests of the
+ Automake testsuite.
+ * tests/tap-functions.sh: New file containing definition of
+ helper shell functions used by the TAP-producing tests of the
+ Automake testsuite.
+ * tests/defs: Remove definitions of some functions that are
+ now defined in `plain-functions.sh'. Instead, source one of
+ `plain-functions.sh' or `tap-functions.sh', depending on whether
+ `$use_tap' is set to "no" or "yes".
+ (exit trap): Call `late_plan_' if `$use_tap' is true. Also,
+ unset shell traces before issuing the latest commands, to avoid
+ confusing the tap driver with spurious output.
+ * tests/defs-static.in ($use_tap): New variable, by default set
+ to "yes" if the calling test script has a `.tap' suffix, and to
+ "no" otherwise. The individual scripts can override it though.
+ In code sanity-checking the environment, verify that `$use_tap'
+ is not exported.
+ * tests/self-check-env-sanitize.test: Update, and small related
+ reformatting.
+ * tests/self-check-tap.test: New very minimal self test.
+ * tests/acoutbs.test, tests/acoutbs2.test, tests/acoutnoq.test,
+ tests/acoutpt.test, tests/acoutpt2.test, tests/acoutqnl.test:
+ Removed, merged into ...
+ * tests/ac-output-old.tap: ... this new test, that uses TAP.
+ * tests/Makefile.am (TAP_LOG_DRIVER): Define to invoke our own
+ `tap-driver' script.
+ (TAP_LOG_DRIVER_FLAGS): Define to `--merge', so that the stdout
+ and stderr of the test scripts remains synced.
+ (TEST_EXTENSIONS): Define, to add the `.tap' suffix; also list
+ the `.test' suffix explicitly.
+ (EXTRA_DIST): Distribute the new files `plain-functions.sh' and
+ `tap-functions.sh'.
+ ($(TEST_LOGS)): Depends on them.
+ (AM_TESTS_ENVIRONMENT): Ensure that `use_tap' is not exported in
+ the environment of the tests.
+ (TESTS): Update.
-2011-06-23 Stefano Lattarini <stefano.lattarini@gmail.com>
+2011-08-03 Stefano Lattarini <stefano.lattarini@gmail.com>
- tests: fix bug in 'autodist.test'
- * tests/autodist.test: Avoid spurious failure due to no
- `defs-static' file being found in the parent directory.
+ * NEWS: Fix typo, and related reformatting.
-2011-06-21 Stefano Lattarini <stefano.lattarini@gmail.com>
+2011-08-03 Stefano Lattarini <stefano.lattarini@gmail.com>
- maintcheck: extend 'sc_tests_plain_*' checks
- * Makefile.am (sc_tests_plain_autom4te): New check.
- (sc_tests_plain_autoreconf): Likewise.
- (sc_tests_plain_autoheader): Likewise.
- (syntax_check_rules): Update.
+ * NEWS: add mention of new experimental TAP support
-2011-06-21 Stefano Lattarini <stefano.lattarini@gmail.com>
+2011-08-02 Stefano Lattarini <stefano.lattarini@gmail.com>
- tests: interactions between TESTS_ENVIRONMENT and LOG_COMPILER
- * tests/tests-environment-and-log-compiler.test: New test,
- checking that we can use variables and functions set by
- TESTS_ENVIRONMENT and AM_TESTS_ENVIRONMENT in LOG_COMPILER
- and LOG_FLAGS (for tests both with and without registered
- extensions).
- * tests/Makefile.am (TESTS): Update.
+ testsuite: rename some test scripts
+ * tests/parallel-tests-am_tests_environment.test: Renamed ...
+ * tests/am-tests-environment.test: ... to this.
+ * tests/check-tests_environment.test: Renamed ...
+ * tests/tests-environment.test: ... to this.
+ * tests/parallel-test-driver-install.test: Renamed ...
+ * tests/parallel-tests-driver-install.test: ... to this.
+ * tests/parallel-tests-make-n.test: Renamed ...
+ * tests/parallel-tests-dry-run.test: ... to this.
+ * tests/Makefile.am (TESTS): Updated.
-2011-06-20 Stefano Lattarini <stefano.lattarini@gmail.com>
+2011-08-02 Stefano Lattarini <stefano.lattarini@gmail.com>
- maintcheck: avoid few spurious failures
- * Makefile.am (sc_tests_plain_aclocal, sc_tests_plain_perl,
- sc_tests_plain_autoconf, sc_tests_plain_automake,
- sc_tests_plain_autoupate): Be stricter in matching an erroneous
- literal command, i.e., `aclocal', `automake', `perl', etc.
+ coverage: parallel-tests and lazy dependencies on EXTRA_PROGRAMS
+ * tests/parallel-tests-extra-programs.test: New test.
+ * tests/Makefile.am (TESTS): Update.
+ * doc/automake.texi (Parallel Test Harness): Add a comment
+ pointing to the new test.
+
+2011-08-02 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ docs: improve, extend and fix documentation on TAP support
+ * doc/automake.texi ("Using the TAP test protocol"): Divide this
+ section into ...
+ ("Introduction to TAP", "Use TAP with the Automake test harness",
+ "Incompatibilities with other TAP parsers and drivers", "Links
+ and external resources"): ... these subsections, extend them by
+ adding more information and examples, and improve them by removing
+ incomplete and/or temporary wordings and TODO items.
+ ("Script-based Testsuites", "Parallel Test Harness"): Add a couple
+ of anchors to improve the granularity of cross-references.
+ * tests/tap-doc2.test: New test, verifying the correctness of the
+ new examples given in the manual.
+ * tests/Makefile.am (tap_other_tests): Add the new test.
+
+2011-08-01 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: separate the only failing check of an xfailing test
+ * tests/plan-bad-prog.test: Move the only failing check of this
+ test (i.e., the one about the total number of "ERROR" outcomes)
+ into ...
+ * tests/plan-bad-prog2.test: ... this new test.
+ * tests/Makefile.am (XFAIL_TESTS): Remove `plan-bad-prog.test',
+ add `plan-bad-prog2.test'.
+ (tap_other_tests): Add `plan-bad-prog2.test'.
+ (plan-bad-prog2.log): Depend on `plan-bad-prog.test'.
+
+2011-08-01 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: fix spurious errors in an xfailing test on TAP support
+ * tests/plan-bad-prog.test: Fix typo in the name of the test
+ being made unreadable. Correct the wording of the potential
+ skip message, and simplify the condition under which the test
+ is to be skipped. Escape literal dots in grep regexps.
+
+2011-08-01 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: better granularity in a couple of tests on TAP support
+ * tests/tap-summary-aux.sh: New auxiliary script, filled with code
+ moved out from ...
+ * tests/tap-summary.test: ... this test, from which the checks on
+ colored testsuite have further been moved into ...
+ * tests/tap-summary-color.test: ... this new test, which in turn
+ * tests/Makefile.am (tap_other_tests): Add `tap-summary-color.test'.
+ (EXTRA_DIST): Distribute `tap-summary-aux.sh'
+ (tap-summary.log): Depend on `tap-summary-aux.sh'.
+ (tap-summary-color.log): Likewise.
+
+2011-08-01 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: optimize tests on TAP for speed and against duplication
+ The creation and configuration of common files and data used by
+ many tests on TAP is, with this change, factored out into a new
+ dedicated auxiliary test, from which those tests will depend upon.
+ This reduces code duplication in tests and, more importantly,
+ offers a noticeable speedup in the involved tests (30-40%).
+ * tests/tap-common-setup.test: New test, setting up the common
+ files and data used by various tests on TAP support.
+ * tests/tap-setup.sh: New file, to be sourced by tests wanting to
+ bring in data generated by `tap-common-setup.test'.
+ * tests/tap-autonumber.test: Update to use the pre-computed data
+ files.
+ * tests/tap-bailout.test: Likewise.
+ * tests/tap-color.test: Likewise.
+ * tests/tap-deps.test: Likewise.
+ * tests/tap-diagnostic.test: Likewise.
+ * tests/tap-empty-diagnostic.test: Likewise.
+ * tests/tap-empty.test: Likewise.
+ * tests/tap-escape-directive.test: Likewise.
+ * tests/tap-exit.test: Likewise.
+ * tests/tap-signal.test: Likewise.
+ * tests/tap-fancy.test: Likewise.
+ * tests/tap-fancy2.test: Likewise.
+ * tests/tap-global-log.test: Likewise.
+ * tests/tap-global-result.test: Likewise.
+ * tests/tap-html.test: Likewise.
+ * tests/tap-log.test: Likewise.
+ * tests/tap-merge-stdout-stderr.test: Likewise.
+ * tests/tap-no-merge-stdout-stderr.test: Likewise.
+ * tests/tap-message-0.test: Likewise.
+ * tests/tap-no-disable-hard-error.test: Likewise.
+ * tests/tap-no-spurious-summary.test: Likewise.
+ * tests/tap-no-spurious.test: Likewise.
+ * tests/tap-not-ok-skip.test: Likewise.
+ * tests/tap-numeric-description.test: Likewise.
+ * tests/tap-out-of-order.test: Likewise.
+ * tests/tap-passthrough.test: Likewise.
+ * tests/tap-passthrough-exit.test: Likewise.
+ * tests/tap-plan.test: Likewise.
+ * tests/tap-plan-corner.test: Likewise.
+ * tests/tap-plan-corner2.test: Likewise.
+ * tests/tap-plan-errors.test: Likewise.
+ * tests/tap-realtime.test: Likewise.
+ * tests/tap-recheck-logs.test: Likewise.
+ * tests/tap-skip-whole.test: Likewise.
+ * tests/tap-todo-skip-together.test: Likewise.
+ * tests/tap-todo-skip-whitespace.test: Likewise.
+ * tests/tap-todo-skip.test: Likewise.
+ * tests/tap-unplanned.test: Likewise.
+ * tests/tap-with-and-without-number.test: Likewise.
+ * tests/tap-xfail-tests.test: Likewise.
+ * tests/tap-skip-whole-whitespace.test: Likewise, and remove
+ redundant definitions of `$sp' and `$tab' (they are already
+ defined in `tests/defs').
+ * tests/tap-whitespace-normalization.test: Likewise.
+ * tests/Makefile.am (TESTS): Update.
+ (EXTRA_DIST): Distribute `tap-setup.sh'.
+ (tap_with_common_setup_tests, tap_with_common_setup_logs)): New
+ variables, holding respectively the list of tests using the files
+ pre-computed by `tap-common-setup.test', and the list of their
+ corresponding log files.
+ (tap_other_tests): New variable, holding the list of other tests
+ on TAP support.
+ ($(tap_with_common_setup_logs)): Depend on `tap-common-setup.log'
+ and `tap-setup.sh'.
+
+2011-08-01 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ test harness: use new `.trs' files to hold test metadata
+ With this change, the test harness will keep test metadata in
+ dedicated `.trs' files, instead of having them embedded into the
+ `.log' files. This allows for easier forward-compatibility and
+ extension of test metadata, and for more flexibility in the
+ format of the `.log' files. Note that this change makes the
+ `:end-metadata:' field obsolete.
+ * doc/automake.texi (Parallel Test Harness, Log files generation
+ and test results recording): Document the new APIs and behaviour;
+ some related minor rewordings and fixlets.
+ * NEWS: Update.
+ * automake.in (handle_tests): When bringing in the content of
+ `check2.am', substitute %BASE% with the basename of the `.log'
+ file being created by a rule. Add the generated `.trs' files
+ to the list of files to be cleaned by "make mostlyclean".
+ * lib/am/check.am (am__test_driver_flags): Rename ...
+ (am__common_driver_flags): ... to this, and remove the flags
+ `--test-name' and `--log-file' from it: they are now define in
+ the proper rules in `check2.am'.
+ (am__TEST_BASES): New internal variable, holding the names of
+ the tests, with any registered extension removed.
+ (am__stealth_MAKE): New internal variable, can be used instead of
+ $(MAKE) in recipes requiring a recursive call to make, but which
+ are not intended to be executed by "make -n".
+ (.log.trs): New suffix rule, to recover from deletion of `.trs'
+ files.
+ ($(TEST_SUITE_LOG)): Almost completely rewritten to follow the
+ new API of "test logs in `.log' files, test metadata in `.trs'
+ files". It goes to some length to work correctly in face of
+ unreadable or missing `.log' and `.trs' files, and to error out
+ with proper error messages when this is not possible.
+ [%?PARALLEL_TESTS%] (check-TESTS): Also remove relevant "stale"
+ `.trs' files (in addition to `.log files) before remaking the
+ $(TEST_SUITE_LOG).
+ (recheck, recheck-html): Look for the `:recheck:' field in the
+ `.trs' files, not in the `.log' files.
+ * lib/am/check2.am (?GENERIC?%EXT%.log, ?!GENERIC?%OBJ%): Adjust
+ the call to the test driver, in particularly passing the new
+ option `--trs-file'.
+ [%am__EXEEXT%] (?GENERIC?%EXT%$(EXEEXT).log): Likewise.
+ * lib/tap-driver ($USAGE): Adjust the help screen.
+ (Getopt::Long::GetOptions): Handle the `--trs-file' option,
+ through the use of ...
+ ($trs_file): ... this new global variable.
+ (finish): Write metadata for the test run to `$trs_file' rather
+ then to `$log_file', through the use of ...
+ (write_test_results): ... this new function.
+ * lib/test-driver (print_usage): Update the help screen.
+ (Option parsing): Handle the `--trs-file' option, through the
+ use of ...
+ ($resfile): ... this new global variable.
+ (Main code): Write metadata for the test run to `$trsfile' rather
+ than to `$logfile'.
+ Minor related adjustments to comments.
+ * tests/.gitignore: Ignore `*.trs' files.
+ * tests/parallel-tests-unreadable-log.test: Moved ...
+ * tests/parallel-tests-unreadable.test: ... to this, and extended
+ to also check the semantics for unreadable `.trs' files.
+ * tests/test-driver-end-metadata.test: Deleted as obsolete.
+ * tests/test-driver-metadata-no-leading-space.test: Likewise.
+ * tests/test-driver-global-log.test: Renamed ...
+ * tests/test-metadata-global-log.test: ... to this, and modified
+ as to verify the new APIs and semantics.
+ * tests/test-driver-recheck: Renamed ...
+ * tests/test-metadata-recheck.test: ... to this, and modified
+ likewise.
+ * tests/parallel-tests-once.test: New test.
+ * tests/parallel-tests-make-n.test: Likewise.
+ * test-metadata-results.test: Likewise.
+ * test-missing.test: Likewise.
+ * test-missing2.test: Likewise.
+ * test-trs-basic.test: Likewise.
+ * test-trs-recover.test: Likewise.
+ * test-trs-recover2.test: Likewise.
+ * tests/Makefile.am (TESTS): Update.
-2011-06-20 Bert Wesarg <bert.wesarg@googlemail.com> (tiny change)
+2011-07-27 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ test harness: allow more metadata in log files
+ This change reworks and improves the parallel test harness to use
+ more specialized reStructuredText fields in the log files (instead
+ of relying on specially-placed of "magic lines" and more indirect
+ semantical formatting); the new fields are the following:
+ - ":recheck:": tell whether the associated test will have to be
+ re-run by "make recheck";
+ - ":copy-in-global-log:": tell whether the content of the log
+ file should be copied in the "global log" `test-suite.log';
+ - ":end-metadata:", which inhibits the scanning of the rest of
+ the log file (for what concerns test metadata).
+ Also, the special `:test-result:' value "END" has been removed,
+ superseded by the new `:end-metadata:' field.
+ * doc/automake.texi (Log files generation and test results
+ recording): Document the new API and semantics. Remove or fix
+ some obsolete comments.
+ * lib/am/check.am ($(TEST_SUITE_LOG), recheck, recheck-html):
+ Adjust comments and code.
+ * lib/tap-driver (finish): Adjust, with the help of ...
+ (must_recheck, copy_in_global_log): ... these new functions.
+ * lib/test-driver (Main code): Adjust, with the help of ...
+ ($recheck, $gcopy): ... these new variables.
+ * tests/trivial-test-driver: Update to obey the new APIs.
+ * tests/test-driver-recheck.test: Likewise.
+ * tests/test-driver-global-log.test: Likewise.
+ * tests/tap-passthrough.test: Relax the test, avoiding to check
+ what is written in `test-suite.log'; such check has been moved ...
+ * tests/tap-global-log.test: ... in this new test, and extended.
+ * tests/test-driver-metadata-no-leading-space.test: New test.
+ * tests/test-driver-end-test-results.test: Removed, it checked
+ the old APIs; superseded by ...
+ * tests/test-driver-end-metadata.test: ... this new test.
+ * tests/tap-log.test: Improve syncing with ...
+ * tests/test-log.test: ... this new test.
+ * tests/parallel-tests.test: Remove some duplication w.r.t. this
+ last new test. Updated heading comments.
+ * tests/Makefile.am (TESTS): Update.
- check: don't use multi-line coloring for the report
+2011-07-27 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: fix a spurious failure with non-bash shells
+ * tests/tap-fancy2.test: Remove an unportable use of backslashes
+ with the `echo' builtin, which was causing some shells (among them
+ bash and Debian /bin/ksh) to print a `\\' string, while other
+ shells (among them zsh, dash, and Solaris /bin/sh and /bin/ksh)
+ were unexpectedly printing a single `\' character. Since we are
+ at it, add a sanity check to ensure that this issue does not
+ resurface.
+
+2011-07-27 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: fix spurious failures with Solaris /bin/sh
+ * tests/tap-more.test: Use `echo > file', not `: > file', to
+ create empty files in the "for" loops; this is required since,
+ as documented in the autoconf manual, Solaris 10 /bin/sh
+ "optimizes" away the `:' command after the first iteration,
+ even if it is redirected.
+ * tests/test-driver-custom-multitest-recheck2.test: Likewise.
+ * tests/tap-numeric-description.test: Partial rewrite to avoid
+ using positional parameters from the 10th onward, which are
+ unportable to Solaris /bin/sh (using `${10}' causes the shell
+ to die with "bad substitution").
+
+2011-07-31 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ simple tests: support developer-defined fd redirections
+ Motivated by coreutils bug#8846, and related discussions:
+ <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=8846>
+ <http://lists.gnu.org/archive/html/bug-autoconf/2011-06/msg00002.html>
+ In those threads it has been shown how problematic it can be to do
+ portable file descriptor redirections in the testsuite when using
+ the Automake testsuite harness. This change should remedy to that
+ situation.
+ * lib/am/check2.am (?GENERIC?%EXT%$(EXEEXT).log,
+ ?GENERIC?%EXT%.log, ?!GENERIC?%OBJ%): Append redirection defined
+ in $(AM_TESTS_FD_REDIRECT) to the command-line invocations of the
+ test scripts.
+ * lib/am/check.am [!%?PARALLEL_TESTS%] $(check-TESTS): Likewise.
+ * NEWS: Update.
+ * doc/automake.texi (Script-based Testsuites): Document the new
+ feature.
+ * tests/check-fd-redirect.test: New test.
+ * tests/parallel-tests-fd-redirect.test: Likewise.
+ * tests/parallel-tests-am_tests_environment.test: Remove checks
+ about the use of redirections in AM_TESTS_ENVIRONMENT: they would
+ check deprecated (if not undefined) behaviour now. Strengthen a
+ couple of still valid checks, to keep the test more in sync with
+ the documentation. Improve debugging information.
+ * tests/Makefile.am (TESTS): Update.
+
+2011-07-22 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ docs: rework and extend documentation on testsuites support
+ * doc/automake.texi (Generalities about Testing): New section.
+ (Simple Tests using parallel-tests): Section removed, merged
+ into ...
+ (Simple Tests): ... this one, which on the other hand has been
+ subdivided into ...
+ (Script-based Testsuites, Serial Test Harness, Parallel Test
+ Harness): ... these new subsection.
+ (DejaGnu Tests): Minor adjustments.
+ Other related typofixes and rewordings throughout the manual;
+ in particular, avoid to use the term "test driver" for three
+ different concepts (and use instead "test harness" and "test
+ runner" where appropriate).
+ * tests/tap-doc.test: New test.
+ * tests/tap-no-disable-hard-error.test: Likewise.
+ * tests/Makefile.am (TESTS): Update.
+
+2011-07-18 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap: support colorization of testsuite progress output
+ * lib/tap-driver (%COLORS): New variable (definition extracted
+ from `lib/am/check.am:$(am__tty_colors)', with some obvious
+ adjustments.
+ (report): Adjust to colorize console output when required,
+ using ...
+ (decorate_result): ... this new function.
+ (colored): New function, used by the one above.
+ * tests/tap-summary.test: Also run the checks when `color-tests'
+ is in use.
+ * tests/Makefile.am (XFAIL_TESTS): Remove `tap-color.test'.
+
+2011-07-18 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap: some preparatory refactoring (2)
+ This is a follow-up simplification.
+ * lib/tap-driver (console_output): Renamed ...
+ (report): ... to this, and extended to appropriately register
+ the test results when needed.
+ (testsuite_error, handle_tap_comment, handle_tap_test,
+ handle_tap_plan): Adjusted accordingly.
+
+2011-07-18 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap: some preparatory refactoring (1)
+ This refactoring is aimed at simplifying the introduction of
+ colored console output for the TAP driver.
+ * lib/tap-driver (console_output): Now accept two arguments, the
+ first one indicating which kind of thing is to be displayed (for
+ now only a test result or a diagnostic comment), and the second
+ one (if present) the message associated to it.
+ (handle_tap_test, handle_tap_comment, handle_tap_plan,
+ testsuite_error): Adapt to the new `console_output' interface.
+
+2011-07-18 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap: add experimental TAP-aware driver
+ * doc/automake.texi (Using the TAP test protocol): New section.
+ (Overview of Custom Test Drivers Support): Minor updates.
+ * lib/tap-driver: New script, TAP-aware test driver for Automake;
+ implemented in perl and based on TAP::Parser.
+ * lib/Makefile.am (dist_script_DATA): Add it.
+ * tests/tap-autonumber.test: New test.
+ * tests/tap-bailout.test: Likewise.
+ * tests/tap-basic.test: Likewise.
+ * tests/tap-deps.test: Likewise.
+ * tests/tap-diagnostic.test: Likewise.
+ * tests/tap-empty.test: Likewise.
+ * tests/tap-empty-diagnostic.test: Likewise.
+ * tests/tap-escape-directive.test: Likewise.
+ * tests/tap-exit.test: Likewise.
+ * tests/tap-fancy.test: Likewise.
+ * tests/tap-fancy2.test: Likewise.
+ * tests/tap-global-result.test: Likewise.
+ * tests/tap-html.test: Likewise.
+ * tests/tap-log.test: Likewise.
+ * tests/tap-merge-stdout-stderr.test: Likewise.
+ * tests/tap-more.test: Likewise.
+ * tests/tap-more2.test: Likewise.
+ * tests/tap-no-merge-stdout-stderr.test: Likewise.
+ * tests/tap-no-spurious-summary.test: Likewise.
+ * tests/tap-no-spurious.test: Likewise.
+ * tests/tap-not-ok-skip.test: Likewise.
+ * tests/tap-numeric-description.test: Likewise.
+ * tests/tap-out-of-order.test: Likewise.
+ * tests/tap-passthrough.test: Likewise.
+ * tests/tap-plan.test: Likewise.
+ * tests/tap-plan-errors.test: Likewise.
+ * tests/tap-plan-corner.test: Likewise.
+ * tests/tap-realtime.test: Likewise.
+ * tests/tap-recheck-logs.test: Likewise.
+ * tests/tap-recheck.test: Likewise.
+ * tests/tap-skip-whole.test: Likewise.
+ * tests/tap-summary.test: Likewise.
+ * tests/tap-todo-skip.test: Likewise.
+ * tests/tap-todo-skip-together.test: Likewise.
+ * tests/tap-todo-skip-whitespace.test: Likewise.
+ * tests/tap-skipall-whitespace.test: Likewise.
+ * tests/tap-unplanned.test: Likewise.
+ * tests/tap-whitespace-normalization.test: Likewise.
+ * tests/tap-with-and-without-number.test: Likewise.
+ * tests/tap-xfail-tests.test: Likewise.
+ * tests/tap-bad-prog.test: New xfailing test.
+ * tests/tap-color.test: Likewise.
+ * tests/tap-plan-corner2.test: Likewise.
+ * tests/tap-message-0.test: Likewise.
+ * tests/tap-signal.test: Likewise.
+ * tests/Makefile.am (TESTS, XFAIL_TESTS): Update.
+
+2011-07-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ test defs: new auxiliary function 'count_test_results'
+ * tests/defs (count_test_results): New function.
+ * tests/check11.test: Use it.
+ * tests/test-driver-custom-multitest.test: Likewise.
+ * tests/test-driver-custom-multitest-recheck.test: Likewise.
+ * tests/test-driver-custom-multitest-recheck2.test: Likewise.
+ * tests/parallel-tests-log-override-recheck.test: Likewise.
+ * tests/parallel-tests-log-override-recheck.test: Likewise.
+ * tests/parallel-tests-no-spurious-summary.test: Likewise, and
+ slightly improve debugging output.
+ * tests/parallel-tests.test: Make use of `count_test_results'.
+ Also, make grepping of "make check" output slightly stricter
+ * tests/parallel-tests9.test: Likewise.
+ * tests/parallel-tests-log-override-2.test: Likewise, and throw
+ in a small optimization.
+
+2011-07-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ parallel-tests: simplify testsuite summary
+ Prefer a more deterministic, "tabular" format for the testsuite
+ summary, always listing the numbers of passed, failed, xfailed,
+ xpassed, skipped and errored tests, even when these numbers are
+ zero. This simplify the logic of testsuite summary creation,
+ makes it more easily machine-parseable, and will probably allow
+ for easier addition of new kinds of test results in the future.
+ * lib/am/check.am (am__tty_colors_dummy): New make variable, to
+ reduce code duplication. Extracted from previous versions of
+ $(am__tty_colors), and extended by defining two new variables
+ `$mgn' and `$brg'.
+ [%?COLOR%, %!?COLOR%] (am__tty_colors): Use that new variable.
+ (am__text_box): Delete, is not needed anymore.
+ ($(TEST_SUITE_LOG)): Rewrite associated rules to implement the
+ new testsuite summary format.
+ * NEWS: Update.
+ * tests/check10.test: Don't run with the parallel-tests harness
+ too, that makes no sense anymore.
+ * tests/color.test: Update and adjust.
+ * tests/color2.test: Likewise.
+ * tests/parallel-tests.test: Likewise.
+ * tests/parallel-tests3.test: Likewise.
+ * tests/parallel-tests6.test: Likewise.
+ * tests/parallel-tests9.test: Likewise.
+ * tests/parallel-tests-unreadable-log.test: Likewise.
+ * tests/parallel-tests-empty-testlogs.test: Likewise.
+ * tests/parallel-tests-log-override-recheck.test: Likewise.
+ * tests/parallel-tests-no-spurious-summary.test: Likewise.
+ * tests/test-driver-custom-multitest.test: Likewise.
+ * tests/test-driver-end-test-results.test: Likewise.
+ * tests/parallel-tests-no-color-in-log.test: New test.
+ * tests/testsuite-summary-color.test: Likewise.
+ * tests/testsuite-summary-count.test: Likewise.
+ * tests/testsuite-summary-count-many.test: Likewise.
+ * tests/testsuite-summary-reference-log.test: Likewise.
+ * tests/testsuite-summary-checks.sh: New auxiliary script, used
+ by the new tests above.
+ * tests/extract-testsuite-summary: Likewise.
+ * tests/trivial-test-driver: Optimize for speed when there are
+ lots of of tests.
+ * tests/Makefile.am (EXTRA_DIST): Distribute them.
+ (testsuite-summary-color.log, testsuite-summary-count.log): Depend
+ on them.
+ (testsuite-summary-count-many.log): Depend on the auxiliary scripts
+ 'trivial-test-driver' and 'extract-testsuite-summary'.
+ (TESTS): Update.
+
+2011-07-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ parallel-tests: new recognized test result 'ERROR'
+ * lib/am/check.am ($(TEST_SUITE_LOG)): Recognize a new test result
+ `ERROR'. Use it when encountering unreadable test logs (previously
+ a simple `FAIL' was used in this situations).
+ * lib/test-driver: Set the global test result to `ERROR' when the
+ test exit status is 99. When doing colorized output, color `ERROR'
+ results in magenta.
+ * doc/automake.texi (Log files generation and test results
+ recording): Update by listing `ERROR' too among the list of valid
+ `:test-results:' arguments.
+ * NEWS: Update.
+ * tests/trivial-test-driver: Update.
+ * tests/parallel-tests.test: Likewise.
+ * tests/parallel-tests-harderror.test: Likewise.
+ * tests/parallel-tests-no-spurious-summary.test: Likewise.
+ * tests/test-driver-global-log.test: Likewise.
+ * tests/test-driver-recheck.test: Likewise.
+ * tests/test-driver-custom-multitest-recheck.test: Likewise.
+ * tests/test-driver-custom-multitest-recheck2.test: Likewise.
+ * tests/test-driver-custom-multitest.test: Likewise.
+ * tests/test-driver-custom-no-html.test: Likewise.
+ * tests/test-driver-end-test-results.test: Likewise.
+ * tests/color.test: Likewise. Also, make stricter, and also test
+ from VPATH.
+ * tests/color2.test: Likewise, and improve syncing with color.test.
+ * tests/parallel-tests-exit-statuses.test: New test.
+ * tests/parallel-tests-console-output.test: Likewise.
+ * tests/Makefile.am (TESTS): Update.
+
+2011-07-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ parallel-tests: make parsing of test results safer
+ The new code for parsing the testsuite-generated `.log' files,
+ as introduced in commit `v1.11-872-gc96b881', considers each
+ `:test-result:' field anywhere in a `.log' file as a declaration
+ of a test result, and accounts for it as such in the testsuite
+ summary. Unfortunately this could easily cause spurious test
+ failures being reported in the testsuite summary. This happened
+ in practice with the Automake's own testsuite; for example:
+ $ make check TESTS='check12-p.test'; echo exit: $?
+ ...
+ PASS: check12-p.test
+ =====================================
+ 4 of 5 tests failed
+ See tests/test-suite.log
+ Please report to bug-automake@gnu.org
+ =====================================
+ make[2]: *** [test-suite.log] Error 1
+ make: *** [check-am] Error 2
+ exit: 2
+ This change introduces a new special `:test-result:' "END", that,
+ when seen, prevents the rest of the log file from being parsed.
+ For more information, refer to the thread:
+ <http://lists.gnu.org/archive/html/automake-patches/2011-06/msg00199.html>
+ * lib/am/check.am ($(TEST_SUITE_LOG)): Stop the parsing of a log
+ file as soon as the special ":test-result:END" directive is seen.
+ Related changes and enhancements.
+ * lib/test-driver: Protect the rest of the log after the result
+ lined with a ":test-result:END" directive.
+ * doc/automake.texi (Log files generation and test results
+ recording): Update, and related improvements.
+ * tests/parallel-tests-no-spurious-summary.test: New test.
+ * tests/test-driver-end-test-results.test: Likewise.
+ * tests/Makefile.am (TESTS): Update.
+
+2011-06-29 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ docs: document custom test drivers and protocols
+ * doc/automake.texi (Simple Tests): Note that the TESTS_ENVIRONMENT
+ use suggested here is not portable to 'parallel-tests'.
+ (Simple Tests using parallel-tests): Document new restrictions on
+ the uses of TESTS_ENVIRONMENT and AM_TESTS_ENVIRONMENT.
+ (Custom Test Drivers): New section and node.
+ (Overview of Custom Test Drivers Support): New subsection.
+ (Declaring Custom Test Drivers in @file{Makefile.am}): Likewise.
+ (APIs for Custom Test Drivers): Likewise.
+ (Options): Update description of color-tests.
+ * lib/am/check ($(TEST_SUITE_LOG)): Remove comments that have been
+ moved into the manual.
+ (recheck, recheck-html): Minor adjustments to better conform to the
+ documentation (this should cause no semantic changes w.r.t. the
+ former behaviour); minor improvements and extensions to existing
+ comments.
+ * tests/test-driver-create-log-dir.test: New test.
+ * tests/test-driver-strip-vpath.test: Likewise.
+ * tests/test-driver-global-log.test: Likewise.
+ * tests/test-driver-recheck.test: Likewise.
+ * tests/Makefile.am (TESTS): Update.
+
+2011-07-24 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: small enhancement to 'primary3.test'
+ * tests/primary3.test: Also test that the program named `foo.la'
+ has been effectively created.
+
+2011-07-24 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: a small simplification in 'pr8365-remake-timing.test'
+ * tests/pr8365-remake-timing.test: Avoid redundant definition
+ and use of `$save_AUTOCONF' variable.
+
+2011-07-24 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: remove an extra leading blank line
+ * tests/override-conditional-1.test: Remove extra blank line
+ placed before the shebang line.
+
+2011-07-18 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: fix typos, grammaros and other blunders in comments
+ All affected files changed.
+
+2011-07-16 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ test defs: new subroutine 'seq_', simulating GNU seq(1)
+ * tests/defs (seq_): New subroutine.
+ * tests/instmany.test: Use it.
+ * tests/instmany-mans.test: Likewise.
+ * tests/instmany-python.test: Likewise.
+ * tests/self-check-seq.test: New self test.
+ * tests/Makefile.am (TESTS): Update.
+
+2011-07-16 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: remove duplication about testing of config.* aux files
+ * tests/add-missing.test: Also check that the `AC_CANONICAL_SYSTEM'
+ autoconf macro causes the `config.sub' and `config.guess' scripts
+ to be installed by `automake --add-missing'. Since we are at it,
+ fix minor buglets and cosmetic issues in the `check_' function.
+ * tests/hosts.test: Removed, completely subsumed by the previous
+ test now.
+ * tests/Makefile.am (TESTS): Update.
+
+2011-07-16 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: more uses of the 'unindent' subroutine
+ * tests/backcompat.test: Prefer `unindent' over plain `cat' for
+ here documents created from within a loop of if/else body.
+ * tests/backcompat2.test: Likewise.
+ * tests/backcompat4.test: Likewise.
+ * tests/init.test: Likewise.
+ * tests/instmany.test: Likewise.
+ * tests/instmany-mans.test: Likewise.
+ * tests/instmany-python.test: Likewise.
+ * tests/missing5.test: Likewise.
+ * tests/parallel-am2.test : Likewise.
+ * tests/parallel-am3.test : Likewise.
+ * tests/pr307.test: Likewise.
+ * tests/depend4.test: Likewise. Since we are at it, normalize
+ other code formatting.
+ * tests/instfail-java.test: Prefer `echo' over `cat' + here-doc
+ inside a for loop.
+
+2011-07-16 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: avoid '##'-style comments inside recipe commands
+ * tests/autodist.test: Do not whitespace-indent `##' comments when
+ they are embedded in a makefile rule: having them indented is not
+ part of the Automake API, and might cause failures with e.g., Tru64
+ make.
+ * tests/autodist-subdir.test: Likewise.
+ * tests/backcompat.test: Likewise.
+ * tests/backcompat6.test: Likewise.
+ * tests/colon7.test: Likewise.
+ * tests/posixsubst-scripts.test: Likewise.
+ * tests/posixsubst-sources.test: Likewise.
+
+2011-07-16 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: tweak, improve and extend tests on lisp support
+ * tests/lisp2.test: Prettify, and improve debugging output. Make
+ grepping of automake stderr stricter.
+ * tests/lisp7.test: Ensure verbose printing of captured make
+ output. Add trailing `:' command.
+ * tests/lisp8.test: Likewise.
+ * tests/lisp3.test: Likewise. Also, check that `.el' files and
+ compiled `.elc' files get installed by "make install", and
+ uninstalled by "make uninstall".
+ * tests/lisp4.test: Ensure installed `.el' files gets removed by
+ "make uninstall". Related changes. Add trailing `:' command.
+ * tests/lisp5.test: Likewise.
+ * tests/lisp6.test: Use proper m4 quoting in configure.in. Fix
+ use of blank lines, to improve clarity and symmetry. Fix typo in
+ comment. Add trailing `:' command.
+
+2011-07-16 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: drop useless requirement in 'license.test'
+ * tests/license.test: Drop useless requirement "makeinfo". Add
+ an extra blank line, for clarity.
+
+2011-07-08 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests defs: more uses of $top_testsrcdir
+ * tests/autodist-stamp-vti.test: Use `$top_testsrcdir' instead
+ of `$testsrcdir/..'.
+ * tests/repeated-options.test: Likewise.
+ * tests/suffix5.test: Likewise.
+ * tests/vtexi3.test: Likewise.
+
+2011-07-08 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ self tests: fix typo in comment
+ * tests/self-check-exit.test: Fix typo in heading comments.
+
+2011-07-08 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: remove redundant settings of `errexit' shell flag
+ * tests/amhello-binpkg.test: Do not set the `errexit' shell
+ flag, as it is already set by `tests/defs'.
+ * tests/amhello-cflags.test: Likewise.
+ * tests/amhello-cross-compile.test: Likewise.
+ * tests/ansi2knr-deprecation.test: Likewise.
+ * tests/autodist-no-duplicate.test: Likewise.
+ * tests/distcheck-configure-flags-am.test: Likewise.
+ * tests/distcheck-configure-flags-subpkg.test: Likewise.
+ * tests/distcheck-configure-flags.test: Likewise.
+ * tests/distcheck-hook.test: Likewise.
+ * tests/distcheck-hook2.test: Likewise.
+ * tests/parallel-tests-am_tests_environment.test: Likewise.
+ * tests/parallel-tests-harderror.test: Likewise.
+ * tests/parallel-tests-log-compiler-example.test: Likewise.
+ * tests/parallel-tests-log-override-1.test: Likewise.
+ * tests/parallel-tests-log-override-2.test: Likewise.
+ * tests/parallel-tests-log-override-recheck.test: Likewise.
+ * tests/primary-prefix-couples-documented-valid.test: Likewise.
+ * tests/primary-prefix-couples-force-valid.test: Likewise.
+ * tests/primary-prefix-invalid-couples.test: Likewise.
+ * tests/primary-prefix-valid-couples.test: Likewise.
+ * tests/remake-subdir-from-subdir.test: Likewise.
+ * tests/remake-subdir-gnu.test: Likewise.
+ * tests/remake-subdir-long-time.test: Likewise.
+ * tests/remake-subdir.test: Likewise.
+ * tests/remake-subdir2.test: Likewise.
+ * tests/silent-configsite.test: Likewise.
+ * tests/tests-environment-backcompat.test: Likewise.
+
+2011-06-29 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ docs: explain why AM_TESTS_ENVIRONMENT must be semicolon-terminated
+ * doc/automake.texi (Simple Tests using parallel-tests): Ditto, and
+ related adjustments.
+ Suggestion by Ralf Wildenhues.
+
+2011-06-29 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ docs: fix unportable example of AM_TESTS_ENVIRONMENT usage
+ * doc/automake.texi (Simple Tests using parallel-tests): The
+ old example on AM_TESTS_ENVIRONMENT relied on unportable shell
+ features, and in particular didn't work with various Korn
+ Shells (see also commit `v1.11-925-g29ca903'). Give another
+ example, simpler this time, but still inspired to real-world
+ usage (the GNU coreutils testsuite).
+
+2011-06-23 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ docs: avoid a footnote, some related rewordings and improvements
+ * doc/automake.texi (Dist): Reword the part about automatically
+ distributed files to avoid a footnote. Since we are at it, extend
+ a bit, and add an example and a reference to a relevant test case.
+
+2011-06-23 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ docs: minor cosmetic fixes
+ * doc/automake.texi: Break few overly long lines, throughout the
+ file.
+ ("Simple Tests"): Move @vindex for XFAIL_TESTS to the correct
+ position, i.e., before and not after the paragraph where it is
+ introduced.
+ ("Options" @item ansi2knr): Use @pxref instead of @xref. This
+ fixes a texinfo warning.
+ ("Other things Automake recognizes" @item AM_C_PROTOTYPES): Use
+ @pxref instead of @ref.
+
+2011-06-23 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ help: improve text about automatically-distributed files
+ This change fixes automake bug#7819.
+ * automake.in (usage): Distinguish between files that are always
+ automatically distributed when found, and those which are only
+ "under certain conditions".
+ * doc/automake.texi (Basics of Distribution): Update accordingly.
+ * tests/autodist-subdir.test: Update.
+ * tests/autodist-no-duplicate.test: Likewise.
+ * tests/autodist.test: Likewise.
+ (configure.in): Remove useless call to AM_MAINTAINER_MODE.
+
+2011-06-23 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ refactor: split 'usage' subroutine in automake
+ This change is related to automake bug#7819.
+ * automake.in (print_autodist_files): New subroutine,
+ extracted from ...
+ (usage): ... this, which now uses it.
+ * tests/autodist-no-duplicate.test: New test.
+ * tests/Makefile.am (TESTS): Update.
+
+2011-06-23 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: fix bug in 'autodist.test'
+ * tests/autodist.test: Avoid spurious failure due to no
+ `defs-static' file being found in the parent directory.
+
+2011-06-21 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ parallel-tests: allow each test to have multiple results
+ With this change, we improve the code creating the `test-suite.log'
+ global log and the console testsuite summary to make it able to
+ grasp multiple results per test script. This is required in order
+ to introduce the planned support for test protocols, like TAP and
+ SubUnit, which can indeed run multiple tests per test script, each
+ with its individual result.
+ The implementation makes use of a custom reStructuredText field
+ `:test-result:'.
+ Note that no new documentation is added by this change; that is
+ be left for follow-up changes.
+ * lib/check.am ($(TEST_SUITE_LOG)): When processing .log files,
+ recognize a report of a test's result only if it is declared with
+ the custom `:test-result:' reStructuredText field placed at the
+ beginning of a line. Extend and add explanatory comments.
+ (recheck, recheck-html): Add explanatory comments.
+ * lib/test-driver: Write an appropriate reStructuredText field
+ `:test-result:' in the generated log file. Use a reStructuredText
+ transition to better separate the test outcome report from the
+ test script's registered output. Improve comments.
+ * tests/test-driver-custom-xfail-tests.test: Adjust.
+ * tests/parallel-tests7.test: Adjust.
+ * tests/parallel-tests-empty-testlogs.test: New test.
+ * tests/parallel-tests-recheck-override.test: Likewise.
+ * tests/parallel-tests2.test: Extend and keep more in-sync with ...
+ * tests/test-driver-custom-html.test: ... this new related test.
+ * tests/test-driver-custom-no-html.test: New test.
+ * tests/test-driver-custom-multitest.test: Likewise.
+ * tests/test-driver-custom-multitest-recheck.test: Likewise.
+ * tests/test-driver-custom-multitest-recheck2.test: Likewise.
+ * tests/trivial-test-driver: New file, used by the last four tests
+ above.
+ * tests/Makefile.am (TESTS): Update.
+ (EXTRA_DIST): Distribute `trivial-test-driver'.
+ (test-driver-custom-multitest.log): Depend on `trivial-test-driver'.
+ (test-driver-custom-multitest-recheck.log): Likewise.
+ (test-driver-custom-multitest-recheck2.log): Likewise.
+ (test-driver-custom-html.log): Likewise.
+
+2011-06-21 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ parallel-tests: allow custom driver scripts
+ Allow suffix-based definition of custom "driver script" for the
+ test scripts. These driver scripts will be responsible of
+ launching the tests (or their corresponding $(LOG_COMPILER), if
+ they have an associated one), interpreting and displaying the
+ test results, and writing the `.log' files.
+ This new API should allow easy and flexible use of different
+ test protocols in the future; in particular, we plan to use it
+ to implement TAP and SubUnit harnesses.
+ Note that no new documentation is added by this change; that is
+ be left for follow-up changes.
+ * automake.in (handle_tests): Define default for $(LOG_DRIVER),
+ and, for any registered test extension `<ext>', define defaults
+ for $(<ext>_LOG_DRIVER). Substitute %DRIVER% using these new
+ variables, instead of the old internal $(am__test_driver). When
+ processing check2.am, also substitute %DRIVER_FLAGS%.
+ Require auxiliary script `test-driver' only if no driver has been
+ explicitly defined for the test script kinds.
+ * am/check2.am (?GENERIC?%EXT%$(EXEEXT).log, ?GENERIC?%EXT%.log,
+ ?!GENERIC?%OBJ%): Pass the %DRIVER_FLAGS% to the %DRIVER% call.
+ * tests/parallel-tests-no-extra-driver.test: New test.
+ * tests/test-driver-custom.test: Likewise.
+ * tests/test-driver-custom-xfail-tests.test: Likewise.
+ * tests/test-driver-fail.test: Likewise.
+ * tests/Makefile.am: Update.
+ * NEWS: Update.
+
+2011-06-21 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ parallel-tests: add auxiliary script 'test-driver', refactor
+ This refactoring should cause no API of functionality change,
+ and is meant only to simplify the future implementation of TAP
+ and SubUnit testsuite drivers. More precisely, our roadmap is
+ to move most of the "testsuite driving" features out of the
+ Automake-generated Makefiles, and into external scripts with
+ well-defined interfaces. This will allow the user to define
+ its own personalized testsuite drivers, and will also offer us
+ a framework upon which to implement our new TAP and SubUnit
+ drivers, all in a very unobtrusive way and retaining an high
+ degree of code reuse and backward-compatibility.
+ * lib/test-driver: New auxiliary script.
+ * lib/Makefile.am (dist_SCRIPT_DATA): Add it.
+ * automake.in (handle_tests): Require the new auxiliary script
+ `test-driver', and define a new internal makefile variable
+ `$(am__test_driver)', used to call it. Perform new substitution
+ on `DRIVER' when processing the `check2.am' file.
+ * lib/check.am (am__tty_colors): Define new shell variable
+ `$am__color_tests'.
+ (am__rst_section): Removed, its role taken over by the new
+ `test-driver' script.
+ (am__test_driver_flags): New variable, contains the command
+ line options passed to `test-driver'.
+ (am__check_pre): Do not deal with temporary files and exit
+ traps anymore, as the `test-driver' script takes care of that
+ now. Define shell variable `$am__enable_hard_errors', used by
+ `$(am__test_driver_flags)'. Reorder so that we don't need to
+ save and restore the value of the `TERM' environment variable
+ anymore.
+ Other related adjustments.
+ (am__check_post): Remove, as its role has been completely taken
+ over by the `test-driver' script.
+ * am/check2.am (?GENERIC?%EXT%$(EXEEXT).log, ?GENERIC?%EXT%.log,
+ ?!GENERIC?%OBJ%): Call the test script through the Automake
+ substituted `%DRIVER%', and honor the command-line options
+ in `$(am__test_driver_flags)'. Do not call the obsoleted
+ `$(am__check_post)' anymore.
+ * doc/automake.texi (Auxiliary Programs): Mention the new
+ `test-driver' script.
+ (Optional): Mention `test-driver' in AC_CONFIG_AUX_DIR.
+ Since we are at it, break the list of auxiliary scripts by
+ placing one per line, to simplify potential future additions
+ of new scripts.
+ * tests/check.test: Adjust.
+ * tests/check2.test : Likewise.
+ * tests/check3.test : Likewise.
+ * tests/check4.test : Likewise.
+ * tests/check10.test: Likewise.
+ * tests/color.test: Likewise.
+ * tests/color2.test: Likewise.
+ * tests/comment9.test: Likewise.
+ * tests/dejagnu.test: Likewise.
+ * tests/exeext4.test: Likewise.
+ * tests/maken3.test: Likewise.
+ * tests/maken4.test: Likewise.
+ * tests/parallel-tests-interrupt.test: Likewise.
+ * tests/posixsubst-tests.test: Likewise.
+ * tests/repeated-options.test: Likewise.
+ * tests/check-no-test-driver.test: New test.
+ * tests/parallel-test-driver-install.test: Likewise.
+ * tests/Makefile.am (TESTS): Update.
+ * NEWS: Update.
+
+2011-06-21 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ maintcheck: extend 'sc_tests_plain_*' checks
+ * Makefile.am (sc_tests_plain_autom4te): New check.
+ (sc_tests_plain_autoreconf): Likewise.
+ (sc_tests_plain_autoheader): Likewise.
+ (syntax_check_rules): Update.
+
+2011-06-21 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: interactions between TESTS_ENVIRONMENT and LOG_COMPILER
+ * tests/tests-environment-and-log-compiler.test: New test,
+ checking that we can use variables and functions set by
+ TESTS_ENVIRONMENT and AM_TESTS_ENVIRONMENT in LOG_COMPILER
+ and LOG_FLAGS (for tests both with and without registered
+ extensions).
+ * tests/Makefile.am (TESTS): Update.
+
+2011-06-24 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: make 'subst-no-trailing-empty-line.test' more robust
+ * tests/subst-no-trailing-empty-line.test: Strengthen existing
+ checks, to try more scenarios and be slightly stricter in some
+ grepping checks.
+
+2011-06-24 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: fix an incomplete comment
+ * tests/subst-no-trailing-empty-line.test: Fix an incomplete
+ comment, and move it to a more proper place. Minor cosmetic
+ fixes to other comments.
+
+2011-06-20 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ maintcheck: avoid few more spurious failures
+ * tests/depmod-data.test: Use creative quoting to avoid
+ spuriously triggering the `sc_tests_Exit_not_exit' maintainer
+ check. Commit `v1.11-900-g3453b8e' attempted to fix it, but
+ succeeded only partially.
+ * tests/cond33.test: Fix header comments, not to reference
+ obsolescent make variable `$(mkdir_p)'.
+ * tests/cond4.test: Rewrite "$MAKE exp=..." as "exp=.. $MAKE -e",
+ to please `sc_tests_overriding_macros_on_cmdline'
+ * tests/cond19.test: Likewise.
+ * tests/cond32.test: Likewise.
+ * tests/add-missing.test: Use "AUTOMAKE_fails" instead of
+ "$AUTOMAKE ... && Exit 1", to please `sc_tests_automake_fails'.
+
+2011-06-20 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ maintcheck: avoid few spurious failures
+ * Makefile.am (sc_tests_plain_aclocal, sc_tests_plain_perl,
+ sc_tests_plain_autoconf, sc_tests_plain_automake,
+ sc_tests_plain_autoupate): Be stricter in matching an erroneous
+ literal command, i.e., `aclocal', `automake', `perl', etc.
+
+2011-06-20 Bert Wesarg <bert.wesarg@googlemail.com> (tiny change)
+
+ check: don't use multi-line coloring for the report
"less -R" can't handle multi-line coloring as it is done for the
check reports of the serial and parallel testsuite, because of
performance reasons. Thus, color each line of the check report
See also following CC:ed thread on bug-autoconf list:
<http://lists.gnu.org/archive/html/bug-autoconf/2011-06/msg00002.html>
+2011-06-16 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: make test 'self-check-reexec.test' more portable
+ * tests/self-check-reexec.test: Rewrite not to require a Korn
+ Shell able to grok variable expansions such as `${.sh.version}';
+ Solaris 10 /bin/ksh fails on this for example. Instead, just
+ require bash and a shell that is not bash.
+
+2011-06-14 Stefano Lattarini <stefano.lattarini@gmail.com>
+ Jim Meyering <meyering@redhat.com>
+
+ test defs: fix ksh-related portability bug in warning messages
+ Running "make check" normally prints a diagnostic to the outermost
+ stderr (usually a tty) to explain why a test is skipped, thus
+ giving better and faster feedback to the user. It used to do
+ so by redirecting file descriptor 9 to stderr (via "exec 9>&2")
+ before invoking the test scripts, which then would write any skip
+ explanation to file descriptor 9 via the `skip_' function defined
+ in `tests/defs'.
+ However, various Korn Shells (at least Solaris 10's /bin/ksh and
+ Debian GNU/Linux's /bin/ksh) and the HP-UX's /bin/sh close open
+ file descriptors > 2 upon an `exec' system call; thus the effects
+ of "exec 9>&2" are cancelled upon fork-and-exec, so we would get
+ a "Bad file number" diagnostic and no skip explanation with those
+ shells.
+ The present change remedies this situation.
+ * tests/Makefile.am (AM_TESTS_ENVIRONMENT): Redirect more portably,
+ via a trailing "9>&2", rather than the prior "exec 9>&2; ...". Add
+ explanatory comments.
+ * tests/defs (stderr_fileno_): Update the advice in comments.
+ Based on commit v8.12-82-g6b68745 "tests: accommodate HP-UX and
+ ksh-derived shells" in GNU coreutils.
+ Further references, with lots of discussion:
+ <http://lists.gnu.org/archive/html/bug-autoconf/2011-06/msg00002.html>
+ <http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/22488>
+ <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=8846>
+
+2011-06-13 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: remove 'test_prefer_config_shell' from the environment
+ Since commit `v1.11-910-g7df1a9b', the once user-overridable
+ variable `$test_prefer_config_shell' has become an internal
+ detail, and the test scripts now complain and bail out if it is
+ set in the environment.
+ * tests/Makefile.am (AM_TESTS_ENVIRONMENT): Unset the variable
+ `test_prefer_config_shell' if it is set in the environment.
+
+2011-06-13 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: autogenerate list of wrapped tests for `lib/' shell scripts
+ * tests/gen-config-shell-tests: New script, generates distributed
+ makefile snippet `tests/config-shell-tests.am' to list all tests
+ that use the `get_shell_script' function, with names mangled to
+ use suffix `-w.shtst', in ...
+ * tests/Makefile.am (config_shell_tests): ... this macro, whose
+ definition has been consequently removed from Makefile.am.
+ (EXTRA_DIST): Distribute the new script.
+ ($(srcdir)/config-shell-tests.am): Generate using the new script.
+ (include): Include the `config-shell-tests.am' fragment.
+ * bootstrap: Invoke `tests/gen-config-shell-tests' to generate
+ `tests/config-shell-tests.am'.
+ * tests/.gitignore: Ignore `config-shell-tests.am'.
+ * tests/gen-parallel-tests: Fixlet in heading comments.
+
+2011-06-13 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: test mdate-sh with /bin/sh too
+ * tests/mdate5.test: Fetch the `mdate-sh' script using the
+ `get_shell_script' function, and run it directly instead of
+ using `$SHELL'.
+ * tests/mdate6.test: Likewise. Since we are at it, make checks
+ on the `mdate-sh' output stricter, remove now unneeded calls to
+ aclocal and automake and creation/extension of `configure.in',
+ `Makefile.am' and `textutils.tex' files, and add a trailing `:'
+ command.
+ * tests/Makefile.am (config_shell_tests): Add `mdate5-w.shtst'
+ and `mdate6-w.shtst'.
+
+2011-06-13 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: extend tests on `--add-missing' and `--copy' a bit
+ * tests/add-missing.test: Fix typo in heading comments. Try with
+ another testcase that install many (but not all) the auxiliary
+ scripts at once, and uses non-standard (but valid and documented)
+ setups (e.g., defining YACC in Makefile.am instead of calling
+ AC_PROG_YACC from configure.in).
+ * tests/copy.test: Reference `add-missing.test' in heading
+ comments. Try few more test scenarios.
+
+2011-06-16 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: few fixlets and improvements
+ * tests/cond31.test ($required): Remove `cc', it's not really
+ needed.
+ * tests/confh.test: Call autoheader too. The lack of this call
+ wasn't causing spurious failures because the automatic remake
+ rules were somehow invoking it on our behalf (at make time).
+ * tests/fn99subdir.test: Use $subdirname throughout, instead of
+ ${subdirname}, for consistency with the rest of the testsuite.
+ Avoid an unnecessary subshell, which could also cause spurious
+ passes, being guarded by a trailing `|| Exit 1', which neutralize
+ the `errexit' flag. Remove an unnecessary `|| Exit 1' guard.
+ * tests/insh2.test: Rewrite to avoid hackish Makefile.in munging,
+ and to also run configure and make.
+
+2011-06-13 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: don't hard-code test name in txinfo21.test
+ * tests/txinfo21.test: Use `$me' instead of hard-coding the
+ current testcase name "txinfo21". Add a trailing `:' command
+ since we are at it.
+
+2011-06-10 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: new test dedicated to `--add-missing' and `--copy'
+ * tests/add-missing.test: New test.
+ * tests/Makefile.am (TESTS): Update.
+ Suggested by Peter Rosin.
+
+2011-06-08 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: use 'fatal_' and 'framework_failure_' for hard errors
+ * tests/defs (require_xsi): Use `fatal_', not `framework_failure',
+ to report an invalid usage.
+ * tests/remake-gnulib-remove-header.test: Prefer using `fatal_'
+ with a proper error message over a direct call to `Exit 99'.
+ * tests/pr8365-remake-timing.test: Likewise.
+ * tests/cygnus-imply-foreign.test: Likewise.
+ * tests/missing6.test: Likewise.
+ * tests/cond8.test: Likewise.
+ * tests/cond33.test: Likewise.
+ * tests/python-virtualenv.test: Prefer using `framework_failure_'
+ with a proper error message over a direct call to `Exit 99'.
+ * tests/instspc-tests.sh: Prefer using `framework_failure_' and
+ `fatal_' over direct calls to `Exit 99'.
+ (fatal_): Define this (which is a simplified version of the one
+ in `tests/defs') for early uses (i.e., before `tests/defs'
+ gets sourced).
+ * tests/depmode-tests.sh: Likewise. Also, simplify the
+ 'get_depmodes' function and calls to it accordingly.
+
+2011-06-08 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ self tests: check new 'fatal_' function
+ * tests/self-check-exit.test: Also check the new 'fatal_'
+ function.
+
2011-06-08 Stefano Lattarini <stefano.lattarini@gmail.com>
test defs: new function 'fatal_', for hard errors
other reasons. The best way to fix this inconsistency is to
introduce a new function with a more general error message.
Inspired by a recent similar change to Gnulib's tests/init.sh.
- * tests/defs.in (fatal_): New function.
+ * tests/defs (fatal_): New function.
* tests/README (Section "Writing test cases" subsection "Do"):
Suggest the use of `fatal_', not of `framework_failure_', for
generic hard errors. The latter should be reserved for "real"
set-up failures.
-2011-06-05 Stefano Lattarini <stefano.lattarini@gmail.com>
-
- tests: fix typo-related error in auxdir2.test
- * tests/auxdir2.test (configure.in): Close m4 quoting in the
- argument to AC_CONFIG_AUX_DIR. Without this, aclocal fails with
- "ERROR: end of file in string". This problem hasn't been exposed
+2011-06-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: `lib/' shell scripts transparently tested also with $SHELL
+ With the previous commit, the user could prefer the use of $SHELL
+ over /bin/sh in some tests checking the Automake-provided shell
+ scripts by manually exporting `test_prefer_config_shell' to "yes"
+ in the environment. With this commit, we ensure that such tests
+ *always* and *transparently* run using both $SHELL and /bin/sh
+ to execute the checked scripts. The `test_prefer_config_shell'
+ variable becomes an internal detail, and is no more meant to be
+ manually defined or overridden.
+ * tests/defs-static.in: Check that `test_prefer_config_shell' is
+ not exported in the environment. Error out if this is the case.
+ * tests/config-shell-tests.sh: New file, driver script to run
+ checks on the shell scripts in `lib/' using the $SHELL determined
+ at configure time instead of the default system shell /bin/sh.
+ * Makefile.am (TESTS_EXTENSIONS): Add `.shtst'.
+ (SHTST_LOG_COMPILER): Define, it calls `config-shell-tests.sh'.
+ (config_shell_tests): Define to a list of tests that wraps other
+ `*.test' tests using `config-shell-tests.sh'.
+ ($(config_shell_tests)): Dummy dependency declaration required
+ in order to have make actually produce expected log files from
+ the `.shtst.log' suffix rule.
+ (EXTRA_DIST): Distribute `config-shell-tests.sh'.
+ (TESTS): Add `$(config_shell_tests)'.
+ * tests/self-check-env-sanitize.test: Update, by checking that
+ `test_prefer_config_shell' isn't exported in the environment.
+
+2011-06-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: can use also $SHELL to check shell scripts from `lib/'
+ * tests/ar-lib.test: If the variable `$test_prefer_config_shell'
+ is set to "yes", run the script under test with configure-time
+ determined $SHELL, rather than with /bin/sh.
+ The `$test_prefer_config_shell' variable defaults to empty, but
+ can be overridden at runtime by the user, thus allowing more
+ coverage.
+ * tests/compile.test: Likewise.
+ * tests/compile2.test: Likewise.
+ * tests/compile3.test: Likewise.
+ * tests/compile4.test: Likewise.
+ * tests/compile5.test: Likewise.
+ * tests/compile6.test: Likewise.
+ * tests/instsh2.test: Likewise.
+ * tests/instsh3.test: Likewise.
+ * tests/mkinst3.test: Likewise.
+ * tests/missing.test: Likewise.
+ * tests/missing2.test: Likewise.
+ * tests/missing3.test: Likewise.
+ * tests/missing5.test: Likewise.
+ * tests/defs (get_shell_script): New subroutine, factoring out
+ code common to the tests above.
+ (xsi-lib-shell): If `$test_prefer_config_shell' is set to "yes",
+ check that $SHELL, not /bin/sh, supports XSI constructs, as we
+ expect the test will use $SHELL and not /bin/sh to run the
+ script being tested.
+
+2011-06-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests defs: better requirements for XSI shells
+ This change avoids potential spurious failures with tests using
+ the requirement 'xsi-shell' to mean that they want */bin/sh* (not
+ $SHELL) to be XSI-conforming. This idiom used to work before
+ commit `v1.11-874-g1321be7' (as back then the test scripts were
+ unconditionally run with /bin/sh), but has become inconsistent
+ now that the test scripts re-execute themselves with configure
+ determined $SHELL.
+ The described spurious failures have already occurred in practice,
+ for examples on Solaris systems which also had GNU Bash installed.
+ From a suggestion by Peter Rosin. See discussion at:
+ <http://lists.gnu.org/archive/html/automake-patches/2011-06/msg00016.html>
+ * tests/defs (xsi-shell): Now check that $SHELL, rather than the
+ shell currently running the test script, is an XSI shell.
+ (xsi-bin-sh): New requirement, checking that /bin/sh (which can
+ differ from $SHELL) is an XSI shell.
+ (xsi-lib-shell): New requirement, checking that the shell that
+ should be used to test the Automake-provided scripts from `lib/'
+ is an XSI shell. For the moment, this is just an alias for
+ `xsi-bin-sh'.
+ (require_xsi): New subroutine, used to factor out code common to
+ the requirements above.
+ ($xsi_shell_code): New variable, contains shell code supposed to
+ work only with XSI shells. Used by the new subroutine above.
+ * tests/ar-lib.test ($required): Require 'xsi-lib-shell' instead
+ of 'xsi-shell', since the script we test here is run with /bin/sh,
+ not with $SHELL.
+ * tests/compile3.test: Likewise.
+ * tests/compile6.test: Likewise.
+
+2011-06-05 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: fix typo-related error in auxdir2.test
+ * tests/auxdir2.test (configure.in): Close m4 quoting in the
+ argument to AC_CONFIG_AUX_DIR. Without this, aclocal fails with
+ "ERROR: end of file in string". This problem hasn't been exposed
by the testsuite before because this test is in XFAIL_TESTS, so
its failure went unnoticed, even if it was due to a wrong cause.
Bug introduced in commit v1.11-249-g49ac3de.
+2011-06-05 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ self tests: check that `$me' can be overridden
+ * tests/self-check-me.test: Check that `$me' can be overridden
+ before sourcing ./defs, with or without sourcing ./defs-static
+ beforehand, and that this override is honored. Update heading
+ comments.
+
+2011-06-02 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ self tests: fix another spurious failure
+ Our ad-hoc usage of `tests/defs' in the testsuite's self tests
+ stopped working properly when we made the test scripts re-execute
+ themselves with the configure-time $SHELL. Fix this.
+ Fixes a regression introduced by commit 'v1.11-874-g1321be7'.
+ * tests/self-check-exit.test: Export `AM_TESTS_REEXEC' to "no"
+ before running the self tests. This fixes a spurious failure
+ present only when the test was run by hand.
+
+2011-06-02 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ silent-rules tests: fix spurious failures with Sun Studio C++
+ * tests/silentcxx.test: The C++ compiler from Sun Studio is named
+ `CC'. Account for this in our grepping checks on the make output.
+ Since we are at it, throw in a couple of improvements to comments
+ and formatting.
+ * tests/silent-many-generic.test: The C++ compiler from Sun Studio
+ is named `CC', and this can cause spurious failures in our grepping
+ of the make output. Work around this by using a wrapper script
+ around the C++ compiler (generated on the fly), since filtering the
+ make output proved to be too fragile.
+
+2011-06-02 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: fix spurious failure in backcompat2.test on NetBSD
+ * tests/backcompat2.test: Add trailing `:' in the body of a `for'
+ loop, in case the last command there might have an exit status
+ different 0. This work around `set -e' issues in some BSD shells,
+ e.g., NetBSD /bin/ksh.
+
+2011-06-02 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: fix spurious failure in autohdr3.test
+ * tests/autohder3.test (Makefile.am): Let `test' depend on `all',
+ so that config.h header is truly remade.
+ Remove now useless call to `$MAKE' when non-GNU make is in use.
+ Failure introduced by commit `v1.11-895-g5e62b96'.
+
+2011-06-02 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: fix spurious failure of cond29.test on NetBSD
+ * tests/cond29.test: Skip if we cannot safely limit the maximal
+ size of used virtual memory to 20K. This fixes a spurious failure
+ on NetBSD.
+ Bug introduced in commit `v1.11-885-g908d335'.
+
+2011-06-02 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ maintcheck: fix again few more failures
+ * tests/cond29.test: To please the `sc_tests_Exit_not_exit'
+ maintainer check, avoid using `sh -c "exit 0"' where a simple
+ `sh -c :' will do.
+ * tests/depmod-data.test: Use creative quoting to avoid
+ spuriously triggering the `sc_tests_Exit_not_exit' maintainer
+ check.
+
2011-06-02 Stefano Lattarini <stefano.lattarini@gmail.com>
maintcheck: fix some more failures
check.
* tests/depcomp8b.test: Likewise.
+2011-06-02 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: don't require GNU make where it's not strictly needed
+ * tests/autohdr3.test: Rewrite to be stricter when make is GNU
+ make. Drop the requirement of GNU make.
+ * tests/aclocal5.test: Drop GNU make requirement, it's not truly
+ needed. Add reference to similar tests `remake-subdir*.test'.
+ * remake-subdir-gnu.test, remake-subdir-from-subdir.test,
+ remake-subdir.test, remake-subdir2.test: Add reference to
+ each other, and to related test `aclocal5.test'.
+ * tests/aclocal6.test: Drop GNU make requirement, it's not
+ truly needed.
+ * tests/confh6.test: Likewise.
+ * tests/lex3.test: Likewise.
+ * tests/remake11.test: Likewise.
+ * tests/subdir5.test: Likewise.
+ * tests/subdir8.test: Likewise.
+ * tests/werror2.test: Likewise.
+ * tests/conff.test: Likewise, and ensure verbose printing of
+ captured make output.
+ * tests/lex5.test: Tweak so that GNU make is no more required.
+ * tests/version7.test: Likewise.
+ * tests/maken2.test: Add explicative comment for why this test
+ requires GNU make.
+ * tests/maken4.test: Let it run also with BSD makes supporting
+ the `.MAKE' special target.
+ * tests/output6.test: Use proper m4 quoting in configure.in.
+ Expand make macros with one-character name using `$(x)', not
+ `$x', for portability. Move checks in the makefiles, rather
+ than relying on grepping the output from make. Drop the now
+ unneeded GNU make requirement.
+
2011-05-29 Stefano Lattarini <stefano.lattarini@gmail.com>
remake: behave better with non-GNU make in subdirectories
* tests/remake-subdir-from-subdir.test: Likewise.
* tests/Makefile.am (TESTS): Update.
+2011-06-01 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ lex tests: fix spurious failures with Solaris lex
+ * tests/lex-lib.test (foo.l): Avoid empty "rules section", which
+ can confuse Solaris lex.
+ * tests/lex-libobj.test (foo.l): Likewise.
+
+2011-06-01 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ lex tests: do not force the use of flex unconditionally
+ * tests/defs (lex): Act more similarly to what the `yacc'
+ requirement does, i.e., only force the use of flex if the
+ $LEX variable is left unset by the user.
+ (flex): Use `skip_' to skip the test if flex is not found.
+
+2011-06-01 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ lex tests: avoid possible hang; fix and extend
+ * tests/lex3.test (foo.l:yywrap): Return 1, not 0, to avoid hangs.
+ Bug introduced in commit 'v1.11-871-geb147a1'.
+ (Makefile.am): Do not add `@LEXLIB@' to `$(LDADD)', as we define
+ our own `yywrap' function.
+ * tests/lex.test (tscan.l): In `yywrap', return 1, not 0, for
+ consistency with the default flex implementation.
+ * tests/lex-libobj.test (yywrap.c): Likewise.
+ * tests/lex-subobj-nodep.test (s1.l): Likewise.
+ * tests/lexvpath.test (foo.c): Likewise.
+ * tests/silent-lex-gcc (foo.l): Likewise.
+ * tests/silent-lex-generic (foo.l): Likewise.
+ * tests/silent-many-gcc (foo5.l): Likewise.
+ * tests/silent-many-generic (foo5.l): Likewise.
+ * tests/lex-lib.test (mu.c): Likewise.
+ Update heading comments, to refer to ...
+ * tests/lex-lib-external.test: ... this new test, which checks
+ that we can get use the `yywrap' function from a system-wide
+ library, if that's available.
+
+2011-06-01 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: prefer `skip_' over `echo ...; Exit 77'
+ * tests/self-check-cleanup.test: When the test must be skipped,
+ use `skip_ REASON' instead of `echo REASON; Exit 77'. Also,
+ make the skip message shorter and clearer.
+
+2011-06-01 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: fix spurious failures in self tests
+ Our ad-hoc usage of `tests/defs' in the testsuite's self tests
+ stopped working properly when we made the test scripts re-execute
+ themselves with the configure-time $SHELL. Fix this.
+ Fixes a bug introduced by commit 'v1.11-874-g1321be7'.
+ * tests/defs: Only check that we can find the client test script
+ when we must re-execute it.
+ * tests/self-check-cleanup.test: Export `AM_TESTS_REEXEC' to "no"
+ before running the self tests.
+ * tests/self-check-dir.test: Likewise.
+ * tests/self-check-explicit-skips.test: Likewise.
+ * tests/self-check-me.test: Likewise.
+ * tests/self-check-sanity.test: Likewise.
+ * tests/self-check-reexec.test: New test.
+ * tests/Makefile.am (TESTS): Update.
+
+2011-06-01 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: prefer ulimit over timeout in test on a memory-hogging bug
+ * tests/cond29.test: Use 'ulimit' instead of 'timeout' to ensure
+ that automake does not uses up too much resources. This is really
+ and improvement because the bug tested by this script wasn't just
+ a "it hangs" or "it runs too slow" bug, but rather a memory-hogging
+ bug (due to combinatorial explosion when many Automake conditionals
+ had to be handled) which could easily crash the whole system, which
+ is unacceptable. The requirement of a proper and working 'ulimit'
+ builtin might cause the test to be skipped on more systems, but
+ that shouldn't be a problem since the bug isn't about a portability
+ issue, but is rather an automake internal implementation problem.
+ I've verified that the test as updated by this patch still passes
+ with automake 1.8.5, automake 1.10.2, and obviously the development
+ version of automake, and that it fails with automake 1.7.9.
+
+2011-06-01 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: avoid generating `*-p.test' tests, use a wrapper script
+ The generated `*-p.test' tests had already become just thin layers
+ around the corresponding test scripts. This change makes the final
+ step, converting to the use of a generic wrapper script and thus
+ avoiding the extra test generation (similarly to what is done for
+ the `*.instspc' and `*.depmod' tests).
+ * tests/parallel-tests.sh: New file, driver script to run checks
+ the on the `parallel-tests' semantics by wrapping tests that use
+ the generic "Simple Tests" driver.
+ * tests/gen-parallel-tests: Update, mostly to reflect the new
+ `.ptest' extensions used for tests in $(parallel_tests).
+ * Makefile.am (TESTS_EXTENSIONS): Add `.ptest'.
+ (PTEST_LOG_COMPILER): Define, it calls `parallel-tests.sh'.
+ ($(parallel_tests)): Do not really generate `*-p.test' tests
+ anymore; this is now just a dummy dependency declaration required
+ in order to have make actually produce expected log files from
+ the `.ptest.log' suffix rule.
+ (EXTRA_DIST): Distribute `parallel-tests.sh'.
+ (MAINTAINERCLEANFILES): Don't remove the `$(parallel_tests)', it
+ is not necessary anymore.
+ (generated_tests): Variable definition removed.
+ (TESTS): Update, by listing `$(parallel_tests)' directly instead
+ of `$(generated_tests)'
+ (expected_list_of_tests): Remove `$(generated_tests)'.
+ (maintainer-check-list-of-tests): No need to explicitly depend on
+ `$(expected_list_of_tests)' anymore.
+
2011-05-29 Stefano Lattarini <stefano.lattarini@gmail.com>
automake, aclocal: honour configure-time AUTOCONF and AUTOM4TE
the `DESTDIR' variable, is poorly chosen, if not downright wrong.
Rewrite it to use `prefix' as the overridden variable instead.
+2011-05-29 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: don't require gfortran if any fortran compiler is enough
+ * tests/silentf90.test ($required): Require `fortran', not
+ `gfortran'.
+ * tests/silentf77.test ($required): Require `fortran77', not
+ `gfortran'.
+ * tests/silent-many-generic.test ($required): Require `fortran'
+ and `fortran77' rather than `gfortran'.
+
+2011-05-27 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: improve `ccnoco*.test', better cross-compiling support
+ * tests/ccnoco3.test (Mycomp): Use the `$CC' chosen by `tests/defs'
+ instead of forcing `gcc' unconditionally. This ensures better
+ coverage in case of cross-compiling, when GCC can be named e.g.,
+ `i586-mingw32msvc-gcc', instead of simply `gcc'.
+ * tests/ccnoco.test: Likewise. Remove redundant checks. Modernize
+ the created `configure.in'. Run tests both in-tree and in VPATH.
+ Export `CC' to the overridden value only once.
+ * tests/ccnoco2.test: Slightly stricter grepping of automake
+ stderr. Add trailing `:' command.
+
2011-05-27 Stefano Lattarini <stefano.lattarini@gmail.com>
testsuite: each test case depends on `defs-static'
* tests/Makefile.am ($(TEST_LOGS)): Depends on `defs-static' too.
Simplify comments.
+2011-05-27 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: use $SHELL to run tests which are shell scripts
+ * tests/Makefile.am (TEST_LOG_COMPILER): Define so that the
+ configure-time $SHELL is used to run the tests.
+ * tests/defs: Add code to re-execute by default the running
+ test script with configure-time $SHELL. Updated comments.
+ * configure.ac: Check that `set -e' is working for $SHELL,
+ not for /bin/sh.
+ * tests/defs-static.in: Update comments.
+ * tests/README (Supported shells): Updated.
+ (Getting details from failures): Don't tell that tests are
+ run by /bin/sh by default.
+
+2011-05-26 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: split 'subst2.test' to improve modularity and coverage
+ * tests/subst2.test: Test removed, split into ...
+ * tests/programs-primary-rewritten.test: ... this test ...
+ * tests/subst-no-trailing-empty-line.test: ... and this one ...
+ * tests/extra-programs-empty.test: ... and this one.
+ * tests/Makefile.am (TESTS): Update.
+
+2011-05-26 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: tweak and improve tests on Automake conditionals
+ The "avoid the requirement of a working compiler" we refer about
+ below is obtained by disabling automatic dependency tracking and
+ defining `CC' (and `EXEEXT', `OBJEXT', `CXX', etc., if needed) to
+ dummy values, either directly in the `Makefile.am', or by AC_SUBST
+ in `configure.in'.
+ The "cosmetic changes" we refer about below consists in, e.g.,
+ adding a trailing `:' command to a script, using proper m4 quoting
+ in `configure.in' files, adding commands that offer more debugging
+ output, explicitly declaring phony targets as such in Makefiles,
+ making use of idioms more consistent with those employed in other
+ tests, following the GNU formatting standards more closely, etc.
+ * tests/cond.test: Cosmetic changes.
+ * tests/cond3.test: Likewise.
+ * tests/cond10.test: Likewise.
+ * tests/cond14.test: Likewise.
+ * tests/cond15.test: Likewise.
+ * tests/cond17.test: Likewise.
+ * tests/cond40.test: Likewise.
+ * tests/cond13.test: Likewise, and make grepping of the generated
+ `Makefile.in' slightly stricter.
+ * tests/cond6.test: Extend by also running ./configure, make and
+ "make install". Few cosmetic changes.
+ * tests/cond8.test: Likewise.
+ * tests/cond2.test: Do few cosmetic changes. Make grepping of
+ automake stderr stricter.
+ * tests/cond20.test: Likewise.
+ * tests/cond23.test: Likewise.
+ * tests/cond24.test: Likewise.
+ * tests/cond42.test: Likewise.
+ * tests/cond46.test: Likewise.
+ * tests/cond9.test: Move more checks in the `Makefile.am' instead
+ of grepping make's output.
+ * tests/cond38.test: Likewise.
+ * tests/cond11.test: Likewise. Avoid the requirement of a working
+ C compiler.
+ * tests/cond16.test: Likewise.
+ * tests/cond22.test: Likewise.
+ * tests/cond30.test: Likewise.
+ * tests/cond31.test: Likewise.
+ * tests/cond4.test: Likewise, and drop the now-useless GNU make
+ requirement as well.
+ * tests/cond18.test: Likewise.
+ * tests/cond19.test: Likewise.
+ * tests/cond22.test: Likewise.
+ * tests/cond25.test: Add trailing `:' command.
+ * tests/cond26.test: Likewise.
+ * tests/cond27.test: Likewise.
+ * tests/cond28.test: Likewise.
+ * tests/cond29.test: Use `unindent' function for better formatting.
+ Other minor cosmetic changes. Use a `timeout' program (if it's
+ available) to determine whether the script takes too long, instead
+ of just hoping that the user will notice an abnormally long test
+ execution time.
+ * tests/cond33.test: Extend a bit. Fix heading comments. Few more
+ cosmetic changes.
+ * tests/cond37.test: Ensure verbose printing of captured make
+ output. Minor cosmetic changes.
+ * tests/cond34.test: Likewise. Avoid the requirement of a working
+ C compiler. Make grepping of make output slightly stricter.
+ * tests/cond35.test: Quote literal dots in grep regexp. Add extra
+ debugging output. Minor cosmetic changes.
+ * tests/cond36.test: Likewise.
+ * tests/cond41.test: Do not simply check that aclocal fails, but
+ also grep its stderr for the expected error message.
+ * tests/cond43.test: Likewise (but the program being automake).
+ Few cosmetic changes.
+ * tests/condd.test: Avoid the requirement of a working C compiler.
+ * tests/condman3.test: Extend by using more man pages and more
+ manpage sections.
+ * tests/condman.test: Removed, it's completely superseded by
+ `condman3.test'.
+ * tests/Makefile.am (TESTS): Update.
+
+2011-05-25 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ lex tests: avoid spurious failures when LEXLIB isn't found
+ The AC_PROG_LEX Autoconf macro does not diagnose a failure to find
+ the "lex library" expected to provide a `yywrap' function (function
+ which is required to link most lex-generated programs). On the
+ contrary, when all the link attempts (i.e., with `-ll' and `-lfl')
+ fail, configure declares that no lex library is needed, and simply
+ proceeds with the configuration process -- only for the build to
+ possibly fail later, at make time.
+ This behaviour might be (partly) intended; the Autoconf manual
+ reads:
+ ``You are encouraged to use Flex in your sources, since it is
+ both more pleasant to use than plain Lex and the C source
+ it produces is portable. In order to ensure portability,
+ however, you must either provide a function `yywrap' or, if
+ you don't use it (e.g., your scanner has no `#include'-like
+ feature), simply include a `%noyywrap' statement in the
+ scanner's source.''
+ This AC_PROG_LEX behaviour is causing some spurious failures of the
+ Automake testsuite in environments which lack a proper library
+ providing `yywrap' (this happens for example in Linux->MinGW cross
+ compilations). But at this point is clear that a proper workaround
+ is to simply provide a fall-back implementation of `yywrap' in our
+ lexers.
+ * tests/cond35.test: Provide a dummy `yywrap' function.
+ * tests/lex3.test: Likewise.
+ * tests/lexvpath.test: Likewise.
+ * tests/silent-many-gcc.test: Likewise.
+ * tests/silent-many-generic.test: Likewise.
+ * tests/silent-lex-gcc.test: Likewise, and a dummy `main' too.
+ * tests/silent-lex-generic.test: Likewise.
+ * tests/lex-lib.test: New test.
+ * tests/lex-libobj.test: New test.
+ * tests/lex-nowrap.test: New test.
+ * tests/Makefile.am (TESTS): Update.
+ * THANKS: Update.
+ Thanks to Russ Allbery for the suggestion.
+
+2011-05-25 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: require C++ compiler explicitly in tests needing it
+ The list of the test scripts which needed a C++ compiler but
+ failed to require it explicitly has been found by running:
+ $ tests=`grep -lE 'CXX|\.c(c|\+\+|xx|pp)' *.test | tr '\n' ' '`
+ $ am_explicit_skips=yes CXX=false make check TESTS="$tests"
+ and looking for tests that reported FAIL instead of SKIP.
+ After the present change, all those previously failing tests
+ either pass or get correctly skipped.
+ * tests/subobj9.test: Require `c++', instead of explicitly
+ skipping on configure failure. Add excerpts and/or details from
+ the original bug report that prompted this tests to be written.
+ * tests/silentcxx-gcc.test: Require g++.
+ * tests/silentcxx.test: Require c++.
+ * tests/suffix3.test: Likewise.
+
+2011-05-25 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: don't require g++ where any C++ compiler is enough
+ * tests/specflg10.test ($required): Use `c++', not g++.
+ * tests/silent-many-generic.test: Likewise.
+
+2011-05-25 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: require C compiler explicitly in tests needing it
+ The list of the test scripts which needed a C compiler but
+ failed to require it explicitly has been found by running:
+ $ am_explicit_skips=yes CC=false make check
+ and looking for tests that reported FAIL instead of SKIP.
+ After the present change, all those previously failing tests
+ either pass or get correctly skipped.
+ * tests/aclocal4.test ($required): Add `cc'.
+ * tests/ansi.test: Likewise.
+ * tests/ansi10.test: Likewise.
+ * tests/ansi3.test: Likewise.
+ * tests/ansi3b.test: Likewise.
+ * tests/ansi5.test: Likewise.
+ * tests/ansi6.test: Likewise.
+ * tests/ansi7.test: Likewise.
+ * tests/ansi9.test: Likewise.
+ * tests/backcompat6.test: Likewise.
+ * tests/canon7.test: Likewise.
+ * tests/check5.test: Likewise.
+ * tests/check7.test: Likewise.
+ * tests/check8.test: Likewise.
+ * tests/cond4.test: Likewise.
+ * tests/cond11.test: Likewise.
+ * tests/cond16.test: Likewise.
+ * tests/cond18.test: Likewise.
+ * tests/cond19.test: Likewise.
+ * tests/cond22.test: Likewise.
+ * tests/cond30.test: Likewise.
+ * tests/cond31.test: Likewise.
+ * tests/cond32.test: Likewise.
+ * tests/cond34.test: Likewise.
+ * tests/cond35.test: Likewise.
+ * tests/cond39.test: Likewise.
+ * tests/condd.test: Likewise.
+ * tests/cygnus-dependency-tracking.test: Likewise.
+ * tests/depcomp2.test: Likewise.
+ * tests/depcomp6.test: Likewise.
+ * tests/depcomp7.test: Likewise.
+ * tests/depcomp8a.test: Likewise.
+ * tests/depcomp8b.test: Likewise.
+ * tests/depdist.test: Likewise.
+ * tests/depend2.test: Likewise.
+ * tests/depend5.test: Likewise.
+ * tests/distcleancheck.test: Likewise.
+ * tests/distname.test: Likewise.
+ * tests/exeext.test: Likewise.
+ * tests/exeext4.test: Likewise.
+ * tests/extradep.test: Likewise.
+ * tests/extradep2.test: Likewise.
+ * tests/gnits2.test: Likewise.
+ * tests/gnits3.test: Likewise.
+ * tests/instdir-ltlib.test: Likewise.
+ * tests/instdir-prog.test: Likewise.
+ * tests/instfail.test: Likewise.
+ * tests/instfail-libtool.test: Likewise.
+ * tests/lex3.test: Likewise.
+ * tests/lex5.test: Likewise.
+ * tests/lexvpath.test: Likewise.
+ * tests/lex-subobj-nodep.test: Likewise.
+ * tests/lflags.test: Likewise.
+ * tests/libobj-basic.test: Likewise.
+ * tests/libobj2.test: Likewise.
+ * tests/libobj7.test: Likewise.
+ * tests/libobj10.test: Likewise.
+ * tests/libobj16a.test: Likewise.
+ * tests/libobj16b.test: Likewise.
+ * tests/libobj17.test: Likewise.
+ * tests/libobj19.test: Likewise.
+ * tests/libtool2.test: Likewise.
+ * tests/libtool3.test: Likewise.
+ * tests/libtool7.test: Likewise.
+ * tests/libtool9.test: Likewise.
+ * tests/libtoo10.test: Likewise.
+ * tests/libtoo11.test: Likewise.
+ * tests/ltcond.test: Likewise.
+ * tests/ltcond2.test: Likewise.
+ * tests/ltconv.test: Likewise.
+ * tests/ltinit.test: Likewise.
+ * tests/ltlibsrc.test: Likewise.
+ * tests/ltorder.test: Likewise.
+ * tests/nobase.test: Likewise.
+ * tests/nobase-libtool.test: Likewise.
+ * tests/mmodely.test: Likewise.
+ * tests/parallel-tests5.test: Likewise.
+ * tests/parallel-tests-suffix-prog.test: Likewise.
+ * tests/parallel-tests-ext-driver-prog.test: Likewise.
+ * tests/posixsubst-ldadd.test: Likewise.
+ * tests/posixsubst-libraries.test: Likewise.
+ * tests/posixsubst-ltlibraries.test: Likewise.
+ * tests/posixsubst-programs.test: Likewise.
+ * tests/posixsubst-sources.test: Likewise.
+ * tests/pr87.test: Likewise.
+ * tests/pr204.test: Likewise.
+ * tests/pr224.test: Likewise.
+ * tests/pr300-lib.test: Likewise.
+ * tests/pr300-ltlib.test: Likewise.
+ * tests/pr300-prog.test: Likewise.
+ * tests/pr401.test: Likewise.
+ * tests/pr401b.test: Likewise.
+ * tests/pr401c.test: Likewise.
+ * tests/remake-gnulib-add-acsubst.test: Likewise.
+ * tests/remake-gnulib-add-header.test: Likewise.
+ * tests/regex.test: Likewise.
+ * tests/repeated-options.test: Likewise.
+ * tests/silent.test: Likewise.
+ * tests/silent3.test: Likewise.
+ * tests/silent9.test: Likewise.
+ * tests/silent-lex-generic.test: Likewise.
+ * tests/silent-many-generic.test: Likewise.
+ * tests/silent-yacc-generic.test: Likewise.
+ * tests/specflg7.test: Likewise.
+ * tests/specflg9.test: Likewise.
+ * tests/specflg10.test: Likewise.
+ * tests/stdinc.test: Likewise.
+ * tests/strip.test: Likewise.
+ * tests/strip2.test: Likewise.
+ * tests/strip3.test: Likewise.
+ * tests/subdirbuiltsources.test: Likewise.
+ * tests/subobj3.test: Likewise.
+ * tests/subobj6.test: Likewise.
+ * tests/subobj11a.test: Likewise.
+ * tests/subpkg.test: Likewise.
+ * tests/subst2.test: Likewise.
+ * tests/subst3.test: Likewise.
+ * tests/substref.test: Likewise.
+ * tests/substre2.test: Likewise.
+ * tests/suffix5.test: Likewise.
+ * tests/suffix8.test: Likewise.
+ * tests/suffix10.test: Likewise.
+ * tests/suffix11.test: Likewise.
+ * tests/suffix12.test: Likewise.
+ * tests/suffix13.test: Likewise.
+ * tests/target-cflags.test: Likewise.
+ * tests/transform.test: Likewise.
+ * tests/transform2.test: Likewise.
+ * tests/yacc-basic.test: Likewise.
+ * tests/yacc-d-basic.test: Likewise.
+ * tests/yacc-clean.test: Likewise.
+ * tests/yacc-dist-nobuild.test: Likewise.
+ * tests/yacc-nodist.test: Likewise.
+ * tests/yaccvpath.test: Likewise.
+ * tests/yacc-d-vpath.test: Likewise.
+ * tests/yacc4.test: Likewise.
+ * tests/yacc7.test: Likewise.
+ * tests/yacc8.test: Likewise.
+ * tests/yaccdry.test: Likewise.
+ * tests/yflags-cmdline-override.test: Likewise.
+ * tests/yflags-force-override.test: Likewise.
+ * tests/python-virtualenv.test: Likewise. Also, improve skip
+ messages.
+ * tests/subobj5.test ($required): Add `cc'.
+ (Makefile.am): Set `AUTOMAKE_OPTIONS' to `subdir-objects', and
+ add new checking rules `test-build' and `test-distdir'.
+ Extend the test by building and examining the distdir, the
+ program, and the object files.
+ * tests/postproc.test ($required): Add `cc'.
+ Avoid the explicit `|| Exit $?' after call to configure, which
+ is now either redundant (e.g., when `am_explicit_skips' is
+ unset), or counter-productive (e.g., when `am_explicit_skips'
+ is set to "yes").
+ * tests/pr243.test: Likewise.
+ * tests/pr266.test: Likewise.
+ * tests/pr220.test: Simplify so that it doesn't require a
+ C compiler anymore.
+ * tests/subdir5.test: Likewise.
+ * tests/subdir8.test: Likewise.
+ * tests/lflags.test: Likewise.
+ * tests/yflags.test: Likewise.
+ * tests/yflags-force-conditional.test: Likewise.
+ * tests/lflags2.test: Simplify so that it doesn't require a
+ C++ compiler anymore.
+ * tests/yflags2.test: Likewise.
+ * tests/autohdrdry.test (configure.in): Remove unneeded call
+ to `AC_PROG_CC'.
+ * tests/pr287.test: Likewise.
+ * tests/check6.test: Likewise.
+ * tests/cond21.test: Likewise, plus some cosmetic adjustments.
+ * tests/upc.test: Skip if configure fails with status `77'.
+ * tests/upc3.test: Likewise.
+ * tests/vala4.test: Likewise.
+ * tests/nostdinc.test: Likewise. Also, make grepping checks
+ on `Makefile.in'. Update heading comments.
+ * tests/compile5.test: When we must skip, skip explicitly,
+ and with a meaningful message.
+ * tests/instspc-tests.sh: When running in "test-build" or
+ "test-install" mode (as determined by the value of variable
+ `$instspc_action'), require `cc'. Adjust comments.
+ * tests/depmod-tests.sh: Likewise.
+
+2011-05-24 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: allow user to ask for stricter skip semantics
+ Given how the Automake testsuite is currently structured, if a
+ command in a test script fails with status `77', the global test
+ outcome is considered a SKIP, because the value of `77' for `$?'
+ is passed to the exit trap. Indeed, this happens in practice, as
+ an autoconf-generated configure script can exit with status `77'
+ if it fails to find, e.g., a required compiler.
+ While this behaviour is quite useful for avoiding spurious test
+ failures in the wild, it can also sometimes prevent the Automake
+ developers to easily see and declare the requirements of their
+ tests.
+ This change introduces a new variable `am_explicit_skips', meant to
+ be user-overridable, and which, when set to a "true" value (i.e.,
+ `yes' or `1'), require a direct call to `Exit 77' in order to make
+ the test outcome be considered a SKIP.
+ * tests/defs.in ($am__test_skipped): New variable, initialized
+ to `no'.
+ (Exit): Set `$am__test_skipped' to `yes' if passed an exit status
+ of 77.
+ (trap '...' 0): When `$am_explicit_skips' is set to a "true" value,
+ reset an exit status of `77' to `78' if $am__test_skipped is not
+ set to `yes'.
+ * tests/self-check-exit.test: Adjust: unset `am_explicit_skips'.
+ * tests/self-check-explicit-skips.test: New test.
+ * tests/Makefile.am (TESTS): Update.
+
+2011-05-24 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: user can force skipping of compiler-requiring tests
+ * tests/defs (cc, c++, fortran, fortran77): Skip the test if
+ the relevant compiler is disabled by having the corresponding
+ variable (CC, CXX, FC and F77, respectively) set to "false".
+ (yacc): For consistency, skip the test when the YACC variable
+ is set to "false", not when it's set to "no". Since we are at
+ it, fix the skip message to be shorter and more consistent.
+
+2011-05-23 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: tweak and improve tests on "Simple Tests" driver
+ * tests/check.test: Make grepping of generated Makefile.in
+ slightly stricter. Add trailing `:' command.
+ * tests/check2.test: Ensure verbose printing of captured make
+ output. Make grepping of generated Makefile.in and of make
+ output stricter. Add trailing `:' command.
+ * tests/check3.test: Likewise. Also, fix minor error in a
+ comment.
+ * tests/check4.test: Stricter grepping of make output. Use
+ `using_gmake' subroutine instead of copy&paste from defs.in
+ to determine whether $MAKE is GNU make. Add a trailing `:'
+ command.
+ * tests/check6.test: Add trailing `:' command.
+ * tests/check10.test: Prefer redirection `>' over `>>' when
+ creating new files. Ensure verbose printing of captured make
+ output.
+ * tests/check11.test: Likewise.
+ * tests/check7.test: Likewise. Make grepping of make output
+ stricter. Prefer "make distcheck" over a simple "make check"
+ in a VPATH build. Add trailing `:' command.
+ (a.c): Format following GNU coding standards.
+ (Makefile.am): Explicitly declare target `print-xfail-tests'
+ as PHONY. Distribute $(check_SCRIPTS), for distcheck.
+ * tests/parallel-tests.test: Prefer redirection `>' over `>>'
+ when creating new files.
+ * tests/parallel-tests2.test: Likewise.
+ * tests/parallel-tests9.test: Likewise.
+ * tests/parallel-tests8.test: Likewise. Make grepping of
+ automake stderr stricter.
+ * tests/check8.test: Likewise. Make formatting of created
+ *.c files more consistent with GNU Coding Standards.
+ * tests/check-subst-prog.test (configure.in): Break overly
+ long lines.
+ (pass-prog.c, xfail-prog.c): Format them more consistently
+ with GNU Coding Standards.
+ * tests/parallel-tests3.test: Use a shorter and clearer skip
+ message. Use the `unindent' subroutine to improve readability
+ of indented loop bodies. Simplify the hack used to please
+ maintainer-check.
+ * tests/parallel-tests5.test: Make formatting of created *.c
+ files more consistent with GNU Coding Standards. Use the
+ `unindent' subroutine to improve readability of indented loop
+ bodies.
+ * tests/parallel-tests10.test: Add blank line, for clarity.
+ Prefer trailing `:' over trailing `Exit 0', for consistency
+ with other tests.
+
+2011-05-21 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests/README: small consistency improvement
+ * tests/README (Section "User interface" subsection "Running the
+ tests"): In examples, be more consistent w.r.t. the differences
+ between GNU and non-GNU make.
+
+2011-05-21 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests/README: suggest how to run tests in cross-compile mode
+ * tests/README (Section "User interface" subsection "Running the
+ tests"): Briefly explain how to override 'host_alias' at runtime
+ to force the use of cross-compilers by the testsuite. Give an
+ example.
+
+2011-05-21 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ test defs: better detection of cross-compile mode
+ * configure.ac (AC_CANONICAL_HOST): New, probably not strictly
+ necessary, but useful to complements AC_CANONICAL_BUILD.
+ * tests/defs-static.in ($host_alias, $build_alias): New variables,
+ exported, user-overridable, defaults AC_SUBST'd by configure.
+ * tests/defs (cross_compiling): Consider cross-compilation mode
+ active only if `$host_alias' is not empty *and* it differs from
+ `$build_alias'. This is more faithful to the way configure
+ determines whether it is cross-compiling, but still allows the
+ user to easily force cross-compilation by overriding `host_alias'
+ and `build_alias' at runtime.
+ From a report by Ralf Wildenhues.
+
2011-05-20 Stefano Lattarini <stefano.lattarini@gmail.com>
testsuite: avoid re-running few tests with 'parallel-tests' option
and 'ANSI2KNR' in the example; instead, use the more common
and typical `DESTDIR'.
+2011-05-18 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: avoid spurious failures in cross-compile mode
+ * tests/depcomp2.test: Ensure verbose printing of captured stderr
+ from configure.
+ * tests/ansi3.test ($required): Add 'native', as the ansi2knr
+ functionality is not meant to work with a cross-compiler.
+ * tests/ansi3b.test: Likewise.
+ * tests/ansi5.test: Likewise.
+ * tests/ansi6.test: Likewise.
+ * tests/ansi7.test: Likewise.
+ * tests/ansi8.test: Likewise.
+ * tests/ansi10.test: Likewise.
+ * tests/subobj3.test: Likewise.
+ * tests/check8.test ($required): Add 'native', as this test is
+ not meant to work with a cross-compiler.
+ * tests/gnits2.test: Likewise.
+ * tests/ltconv.test: Likewise.
+ * tests/ltcond2.test: Likewise.
+ * tests/parallel-tests5.test: Likewise.
+ * tests/specflg7.test: Likewise.
+ * tests/specflg8.test: Likewise.
+ * tests/check5.test: Likewise. Also, ensure verbose printing of
+ captured make output, and thrown in few cosmetic and consistency
+ improvements.
+ * tests/ltinit.test: Be laxer in grepping configure output, to
+ avoid spurious failures on systems which lack POSIX dynamic
+ linking (e.g., MinGW), or when cross-compiling for such systems.
+ * tests/ansi9.test: Skip parts of the test that would require a
+ native compiler.
+ * tests/backcompat6.test: Likewise.
+ * tests/depcomp8a.test: Likewise.
+ * tests/pr401.test: Likewise.
+ * tests/pr401b.test: Likewise.
+ * tests/pr401c.test: Likewise.
+ * tests/target-cflags.test: Likewise.
+ * tests/lex3.test: Likewise.
+ * tests/yacc-basic.test: Likewise.
+ * tests/autohdr4.test: Likewise. Also, extend the test to ensure
+ more coverage in cross-compile mode.
+ * tests/canon7.test: Likewise.
+ * tests/check9.test: Removed, split into two new tests ...
+ * tests/check-subst-prog.test: ... this one, which requires a
+ native C compiler ...
+ * tests/check-subst.test: ... and this one, which doesn't require
+ any compiler.
+ * tests/parallel-tests4.test: Removed, split into two new tests ...
+ * tests/parallel-tests-suffix-prog.test: ... this one, which
+ requires a native C compiler ...
+ * tests/parallel-tests-suffix.test: ... and this one, which
+ doesn't require any compiler.
+ * tests/parallel-tests7.test: Removed, split into two new tests ...
+ * tests/parallel-tests-ext-driver-prog.test: ... this one, which
+ requires a native C compiler ...
+ * tests/parallel-tests-ext-driver.test: ... and this one, which
+ doesn't require any compiler.
+ * tests/Makefile.am (TESTS): Update.
+
2011-05-19 Stefano Lattarini <stefano.lattarini@gmail.com>
test defs: rename requirement 'non-cross' -> 'native'
2011-05-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+ tests: fix spurious failure of extradep.test on FreeBSD
+ * tests/extradep.test: When using `ls -t' to determine whether a
+ file has been updated, make sure to use as reference a file whose
+ timestamp is expected to be *strictly* older that that of the file
+ being checked. This is required because at least FreeBSD `ls' do
+ not sort files with the same timestamp in alphabetical order when
+ using the `-t' option.
+ * tests/extradep2.test: Likewise.
+
+2011-05-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
tests: fix spurious failure of txinfo21.test on FreeBSD
* tests/txinfo21.test: Use the `is_newest' subroutine instead of
the `ls -t' hack to to determine whether a file has been updated.
2011-05-07 Stefano Lattarini <stefano.lattarini@gmail.com>
- tests: fix spurious failure of extradep.test on FreeBSD
- * tests/extradep.test: When using `ls -t' to determine whether a
- file has been updated, make sure to use as reference a file whose
- timestamp is expected to be *strictly* older that that of the file
- being checked. This is required because at least FreeBSD `ls' do
- not sort files with the same timestamp in alphabetical order when
- using the `-t' option.
- * tests/extradep2.test: Likewise.
+ tests: fix syntax error in 'silentcxx-gcc.test'
+ * tests/silentcxx-gcc.test (for config_args in ...): Remove extra
+ trailing backslash from the list of looped-on items.
+
+2011-05-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ depmod tests: fix bug in depmodes list extraction
+ * tests/depmod-tests.sh (get_depmodes): Use `echo $all_depmodes'
+ instead of `... | tr "$nl" "$sp"' to strip extra whitespaces and
+ newlines from `$all_depmodes'. The latter idiom with `tr' is
+ wrong since it gets run before `./defs' is sourced, and thus `$nl'
+ and `$sp' are undefined.
+ Bug revealad by FreeBSD tr(1) implementation, which doesn't accept
+ empty strings as arguments.
+
+2011-05-04 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: add excerpts from few original bug reports, for clarity.
+ * tests/comment6.test: Add excerpts and/or details from the
+ original bug report that prompted this tests to be added and/or
+ extended.
+ * tests/dejagnu4.test: Likewise.
+ * tests/installdir.test: Likewise.
2011-05-04 Stefano Lattarini <stefano.lattarini@gmail.com>
implementation, looking for a generic `lex' program, will follow
in the future.
+2011-05-04 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: 'silent-many-gcc.test' improved and made more robust
+ * tests/silent-many-gcc.test: Also force "fast" gcc depmode
+ for C++ compilations. Add sanity checks verifying that the
+ cache variables we force are really used by configure. Fix
+ typo in comments.
+
+2011-05-04 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: drop useless requirements of 'g++'
+ * tests/lflags2.test ($required): Remove 'g++', as any working
+ C++ compiler should be enough, and ./configure will cause the
+ test to skip if no working C++ compiler is found.
+ * tests/yflags2.test: Likewise.
+ * tests/subobj9.test: Likewise.
+ * tests/silentcxx.test: Likewise. Also, do not force depmodes
+ that could cause non-GNU C++ compilers to fail.
+ * tests/silentcxx-gcc.test: New test, like `silentcxx.test',
+ but forcing "fast" gcc depmode (and thus requiring the GNU C++
+ compiler).
+ * tests/specflg10.test: Add proper "fixme" comment telling that
+ we should make this test work with a generic C++ compiler.
+ * tests/Makefile.am (TESTS): Update.
+
+2011-05-03 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests defs: one more environment sanitization (corner case)
+ * tests/defs (am__using_gmake): Initialize. The `using_gmake'
+ subroutine was using this variable for caching, but wasn't
+ initializing it, which could cause problems in the (admittedly
+ very unlikely) case in which it was pre-existent in the
+ environment.
+
2011-05-03 Stefano Lattarini <stefano.lattarini@gmail.com>
tests: add forgotten test scripts to $(TESTS)
2011-05-03 Stefano Lattarini <stefano.lattarini@gmail.com>
+ maintcheck: consistency of list of test scripts
+ * CheckListOfTests.am: New file.
+ (maintainer-check-list-of-tests): New target,
+ check for consistency between list of tests defined in the
+ including Makefile and list of tests on the filesystem.
+ (clean-maintcheck-testslist-tmp): New rule, to clean up the
+ temporary files that might be left around by the rules associated
+ with the previous target.
+ (clean-local): Depend on it.
+ * lib/Automake/tests/Makefile.am: Include `CheckListOfTests.am'.
+ (maintainer-check): Added dependency from the new target
+ `maintainer-check-list-of-tests'.
+ * tests/Makefile.am: Likewise. Also ...
+ (checked_test_extensions): Define to `.test'.
+ (expected_test_list): Define properly.
+ * Makefile.am (maintainer-check-list-of-test): New target,
+ calling recursively into `tests/' and `lib/Automake/tests/',
+ using ...
+ (TEST_SUBDIRS): ... this new variable.
+
+2011-05-03 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: drop useless requirements of gcc (plus testsuite tweakings)
+ * tests/libobj14.test ($required): Remove 'gcc', as we never run
+ the C compiler.
+ * tests/targetclash.test: Likewise.
+ * tests/ansi6.test ($required): Remove 'gcc', as any working
+ C compiler should be enough.
+ * tests/ansi7.test: Likewise.
+ * tests/ansi9.test: Likewise.
+ * tests/ansi10.test: Likewise.
+ * tests/lex5.test: Likewise.
+ * tests/lexvpath.test: Likewise.
+ * tests/mmodely.test: Likewise.
+ * tests/pr204.test: Likewise.
+ * tests/pr300-lib.test: Likewise.
+ * tests/pr300-prog.test: Likewise.
+ * tests/primary3.test: Likewise.
+ * tests/specflg7.test: Likewise.
+ * tests/specflg8.test: Likewise.
+ * tests/subdir5.test: Likewise.
+ * tests/subdir8.test: Likewise.
+ * tests/subobj6.test: Likewise.
+ * tests/subst3.test: Likewise.
+ * tests/substre2.test: Likewise.
+ * tests/yacc6.test: Likewise.
+ * tests/yacc8.test: Likewise.
+ * tests/depcomp2.test: Likewise. Also, avoid clobbering user-set
+ CFLAGS.
+ * tests/lex3.test: Likewise.
+ * tests/ansi3.test: Likewise. Also, avoid 'CC=gcc' in configure.
+ * tests/ansi3b.test: Likewise.
+ * tests/ansi5.test: Likewise.
+ * tests/autohdr4.test ($required): Remove 'gcc', as any working
+ C compiler should be enough.
+ Also, do not reject slow dependency extractors (which we might
+ be forced to use now that $CC is not necessarily gcc anymore).
+ * tests/cond16.test ($required): Remove 'gcc', as any working
+ C compiler should be enough.
+ Since we are at it, throw in few minor tweakings (mostly cosmetic,
+ stylistic, or consistency-related).
+ * tests/cond18.test: Likewise.
+ * tests/cond35.test: Likewise.
+ * tests/gnits2.test: Likewise.
+ * tests/libtool3.test: Likewise.
+ * tests/libtool7.test: Likewise.
+ * tests/libtool9.test: Likewise.
+ * tests/ltcond.test: Likewise.
+ * tests/ltcond2.test: Likewise.
+ * tests/ltconv.test: Likewise.
+ * tests/ltlibsrc.test: Likewise.
+ * tests/nobase.test: Likewise.
+ * tests/nobase-libtool.test: Likewise.
+ * tests/pr220.test: Likewise.
+ * tests/pr224.test: Likewise.
+ * tests/pr300-ltlib.test: Likewise.
+ * tests/pr401.test: Likewise.
+ * tests/pr401b.test: Likewise.
+ * tests/pr401c.test: Likewise.
+ * tests/subpkg.test: Likewise.
+ * tests/target-cflags.test: Likewise.
+ * tests/transform.test: Likewise.
+ * tests/yacc4.test: Likewise.
+ * tests/cond19.test: Likewise. Also, avoid clobbering user-set
+ CFLAGS.
+ * tests/cond4.test: Likewise.
+ * tests/depend2.test: Likewise.
+ * tests/pr87.test: Likewise.
+ * tests/subobj3.test: Likewise.
+ * tests/substref.test: Likewise.
+
+2011-05-03 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ test defs: increase coverage w.r.t. GNU make
+ * tests/defs (GNUmake): Instead of just skipping the tests
+ requiring GNU make if $MAKE is not GNU make, try to look for
+ it and, if found, redefine $MAKE accordingly. This will help
+ to transparently increase coverage on non-GNU systems which
+ have GNU make available in PATH.
+
+2011-05-03 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: improve few inter-tests references
+ * tests/acloca22.test: Improve and extend the heading comments.
+ Add reference to related tests 'remake-deleted-m4-file.test' and
+ 'remake-renamed-m4-macro-and-file.test'. Since we are at it,
+ add trailing `:' command, and few blank lines for readability.
+ * tests/remake-renamed-m4-macro-and-file.test: In the heading
+ comments, add reference to the related test 'acloca22.test'.
+
+2011-05-03 Stefano Lattarini <stefano.lattarini@gmail.com>
+
tests: fix spurious failure in 'color2.test'
* tests/color2.test: If $MAKE contains command-line arguments (as
in e.g., "make -j2"), expect's directive "spawn $env(MAKE)" fails
spuriously, because it tries to run "$MAKE" as a single command.
Fix this with proper uses of the TCL `eval' builtin.
+2011-05-03 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: few more checks on automatic remake support
+ * tests/remake-all-1.test: New test, check that the "all" target
+ triggers rebuilt of outdated Makefiles.
+ * tests/remake-all-2.test: Likewise, but for when the makefiles
+ are not named `Makefile'.
+ * tests/Makefile.am (TESTS): Update.
+
+2011-05-03 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: check some remake features with non-GNU make too
+ * tests/acloca14.test ($required): Drop "GNUmake". This test
+ works as-is with non-GNU make implementations.
+ * tests/remake1a.test: Likewise.
+ * tests/aclocal4.test ($required): Drop "GNUmake".
+ Modify the test to have it work also with non-GNU make.
+ * tests/remake5.test: Likewise.
+ * tests/remake8a.test: Likewise.
+ * tests/remake8b.test: Likewise.
+ * tests/remake9a.test: Likewise.
+ * tests/remake9b.test: Likewise.
+ * tests/remake9c.test: Likewise.
+ * tests/remake9d.test: Likewise.
+ * tests/remake10a.test: Likewise.
+ * tests/remake10b.test: Likewise.
+ * tests/remake10c.test: Likewise.
+ * tests/remake12.test: ($required): Drop "GNUmake".
+ Adapt the test to make it work also with non-GNU make (if
+ it supports an "include" directive).
+
+2011-05-01 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: extend tests on dependency tracking with VPATH
+ * tests/depmod-tests.sh: New script, fulfilling a threefold role:
+ 1. it is called to generate a Makefile.am snippet, containing the
+ definition of proper lists of tests;
+ 2. it is called to set up a directory containing some common data
+ files and autotools-generated files used by the aforementioned
+ tests (this is done for speed reasons only); and
+ 3. it is called to properly run those tests, one at a time.
+ * tests/depcomp9.test: Delete, it's obsolete now.
+ * tests/depcomp10.test: Likewise.
+ * tests/Makefile.am ($(srcdir)/depmod-tests.am): Include this
+ snippet, which defines ...
+ (depmode_tests): ... this new macro, containing the list of the
+ newly generated `*.depmod' tests.
+ (TESTS_EXTENSIONS): Add `.depmod'.
+ (DEPMOD_LOG_COMPILER): Define. It calls `depmod-tests.sh', so that
+ the "depmode tests" will be executed by passing that driver script
+ a proper parameter.
+ ($(depmod_tests)): New dependency declaration (dummy, but required
+ in order to have make actually produce expected log files from the
+ `.depmod.log' suffix rule).
+ (TESTS): Add $(depmod_tests).
+ (EXTRA_DIST): Distribute depmod-tests.sh.
+ Other minor cosmetic changes and reorderings.
+ * bootstrap: Generate depmod-tests.am.
+ * tests/.gitignore: Updated.
+
+2011-05-01 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: avoid extra test generation (for `instspc' tests)
+ We don't really need to generate tons and tons of wrapper test
+ scripts (presently, ~ 90) just to run what is basically a single
+ test (specifically, `instspc-tests.sh') with different parameters.
+ In fact, the possibility of running a single driver script with
+ multiple files/configurations/parameters is one of the major
+ selling points of the 'parallel-tests' testsuite driver of
+ automake. So why not use ourselves more extensively the best
+ features we provide?
+ * tests/instspc-tests.sh: Update heading comments. Expect
+ to be executed, not sourced. Update handling of command-line
+ arguments. Adjust to reflect the renaming of "instspc tests"
+ from `instspc-*.test' to `*.instspc'.
+ * tests/instspc-data.test: Update heading comments. Execute
+ driver script `instspc-tests.sh', instead of sourcing it.
+ Give more informative messages in case of internal errors.
+ * tests/Makefile.am (TESTS_EXTENSIONS): Define explicitly (to
+ `.test' and `.instspc').
+ (TESTS): Add `$(instspc_tests)'.
+ (generated_tests): Remove `$(instspc_tests)'.
+ (INSTSPC_LOG_COMPILER): Define. Calls `instspc-tests.sh', so
+ that the "instspc tests" will be now executed by passing that
+ driver script a proper parameter.
+ ($(instspc_tests)): Remove, we don't need anymore to generate
+ this tests.
+ ($(instspc_tests:.test=.log)): Remove, substituted by ...
+ ($(instspc_tests:.instspc=.log)): ... this.
+ ($(instspc_tests)): New dependency declaration (dummy, but
+ required in order to have make actually produce expected log
+ files from the `.instspc.log' suffix rule).
+ (MAINTAINERCLEANFILES, generated_tests): Don't extend with
+ $(instspc_tests) anymore.
+ Update comments.
+
+2011-05-01 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ build: improve the definition of the list of testcases
+ * tests/Makefile.am (handwritten_tests): New variable.
+ (generated_tests): Likewise.
+ (TESTS): Redefine as the union of the above.
+ (EXTRA_DIST): Extend using $(handwritten_tests) and
+ $(generated_tests) rather than $(TESTS).
+ * tests/gen-parallel-tests: Update accordingly, and
+ make more robust.
+
+2011-05-01 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests skips: shorter and clearer messages
+ * tests/defs: Use shorter messages when giving reasons for test
+ skipping; it turns out these shorter messages are also clearer.
+ If more info might be useful, send them to the log file only.
+
2011-05-01 Stefano Lattarini <stefano.lattarini@gmail.com>
tests: fix self check spurious failure with older bash versions
and 3.2.39) fail to correctly remove the temporary directory in
the exit trap.
+2011-05-01 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: add sanity check to 'self-check-cleanup.test'
+ * tests/self-check-cleanup.test: Check that the "hacked" file
+ `defs-static' used by the test differs from the "vanilla" one
+ in builddir. This also offers a little more debugging output.
+
+2011-05-01 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: display reasons for skips to the console
+ * tests/Makefile.am (AM_TESTS_ENVIRONMENT): Open file descriptor
+ `9' to the original stderr; define `stderr_fileno_' to `9', and
+ export it.
+ * tests/self-check-report.test: Prevent new spurious failures by
+ removing from the environment any definition of `stderr_fileno_'.
+
+2011-05-01 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: use `skip_' for skipping of tests
+ * tests/defs: Use the `skip_' subroutine for test skipping. Also
+ give sometimes more detailed messages about the reasons for the
+ skipping.
+ * tests/canon7.test: Likewise.
+ * tests/color.test: Likewise.
+ * tests/color2.test: Likewise.
+ * tests/compile2.test: Likewise.
+ * tests/dejagnu7.test: Likewise.
+ * tests/depcomp6.test: Likewise.
+ * tests/depcomp9.test: Likewise.
+ * tests/depcomp10.test: Likewise.
+ * tests/distlinks.test: Likewise.
+ * tests/distlinksbrk.test: Likewise.
+ * tests/fn99.test: Likewise.
+ * tests/fn99subdir.test: Likewise.
+ * tests/forcemiss2.test: Likewise.
+ * tests/fort5.test: Likewise.
+ * tests/gettext3.test: Likewise.
+ * tests/install2.test: Likewise.
+ * tests/instfail-info.test: Likewise.
+ * tests/instfail-java.test: Likewise.
+ * tests/instfail-libtool.test: Likewise.
+ * tests/instfail.test: Likewise.
+ * tests/instmany-mans.test: Likewise.
+ * tests/instmany-python.test: Likewise.
+ * tests/instmany.test: Likewise.
+ * tests/instsh3.test: Likewise.
+ * tests/ltinit.test: Likewise.
+ * tests/makej2.test: Likewise.
+ * tests/mdate6.test: Likewise.
+ * tests/mkinst3.test: Likewise.
+ * tests/parallel-tests3.test: Likewise.
+ * tests/parallel-tests-reset-term.test: Likewise.
+ * tests/parallel-tests-unreadable-log.test: Likewise,
+ * tests/python-virtualenv.test: Likewise.
+ * tests/remake-gnulib-remove-header.test: Likewise.
+ * tests/subobj9.test: Likewise.
+ * tests/symlink2.test: Likewise.
+ * tests/tar.test: Likewise.
+ * tests/tar2.test: Likewise.
+ * tests/txinfo26.test: Likewise.
+ * tests/vala2.test: Likewise.
+ * tests/vala3.test: Likewise.
+ * tests/vala5.test: Likewise.
+ * tests/vtexi4.test: Likewise.
+ * tests/instdir-texi.test: Likewise.
+ * tests/txinfo21.test: Likewise.
+
2011-04-29 Stefano Lattarini <stefano.lattarini@gmail.com>
testsuite: more environment sanitization
--- /dev/null
+## -*- Automake -*-
+##
+## Copyright (C) 2011 Free Software Foundation, Inc.
+##
+## This program is free software; you can redistribute it and/or modify
+## it under the terms of the GNU General Public License as published by
+## the Free Software Foundation; either version 2, or (at your option)
+## any later version.
+##
+## This program is distributed in the hope that it will be useful,
+## but WITHOUT ANY WARRANTY; without even the implied warranty of
+## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+## GNU General Public License for more details.
+##
+## You should have received a copy of the GNU General Public License
+## along with this program. If not, see <http://www.gnu.org/licenses/>.
+##
+
+## Temporary files used in the `check-list-of-tests' target.
+am__tmk = tests-in-makefile-list.tmp
+am__tfs = tests-on-filesystem-list.tmp
+am__tdf = diff-in-tests-lists.tmp
+
+## Check that the list of tests given in the Makefile is equal to the
+## list of all test scripts in the Automake testsuite.
+.PHONY: maintainer-check-list-of-tests
+maintainer-check-list-of-tests:
+## Prefer unified diffs over plain diffs, for readability.
+ @if diff -u /dev/null /dev/null >/dev/null 2>&1; then \
+ diff='diff -u'; \
+ else \
+ diff='diff'; \
+ fi; \
+## Determine which files are to be considered test scripts.
+## Do not blindly look for `$(TEST_EXTENSIONS)', because the parallel
+## testsuite driver provided by Automake allow entries in `$(TESTS)'
+## not to be real files, but rather paramaters for the corresponding
+## LOG_COMPILER (well, sort of).
+ set X $(checked_test_extensions); shift; \
+ if test $$# -eq 0; then \
+ set X $(TEST_EXTENSIONS); shift; \
+ fi; \
+ if test $$# -eq 0; then \
+ echo '$@: internal error: $$(checked_test_extensions) and' \
+ '$$(TEST_EXTENSIONS) are both undefined or empty' >&2; \
+ exit 255; \
+ fi; \
+## List of tests in Makefile.
+ lst='$(expected_list_of_tests)'; \
+ test -n "$$lst" || lst='$(TESTS)'; \
+ for t in $$lst; do \
+ echo "$$t"; \
+ done | sort >$(am__tmk); \
+## List of tests on filesystem. Be careful to cater for VPATH builds too.
+ for ext in $$*; do \
+ ls *$$ext 2>/dev/null; \
+ if test $(srcdir) != $(builddir); then \
+ (cd $(srcdir) && ls *$$ext 2>/dev/null); \
+ fi; \
+ done | sort | uniq >$(am__tfs); \
+## Compare the two lists, complain if they differ.
+ if $$diff $(am__tmk) $(am__tfs) >$(am__tdf); then \
+ result=0; \
+ else \
+ echo '$@: list of tests in Makefile an on filesystem differ' >&2; \
+ echo "+ $$diff in-makefile on-filesystem" >&2; \
+ cat $(am__tdf) >&2; \
+ result=1; \
+ fi; \
+ rm -f $(am__tmk) $(am__tfs) $(am__tdf); \
+ exit $$result;
+
+.PHONY: clean-maintcheck-testslist-tmp
+clean-local: clean-maintcheck-testslist-tmp
+clean-maintcheck-testslist-tmp:
+ rm -f $(am__tmk) $(am__tfs) $(am__tdf)
Installation Instructions
*************************
-Copyright (C) 1994, 1995, 1996, 1999, 2000, 2001, 2002, 2004, 2005,
-2006, 2007, 2008, 2009, 2010 Free Software Foundation, Inc.
+Copyright (C) 1994-1996, 1999-2002, 2004-2011 Free Software Foundation,
+Inc.
Copying and distribution of this file, with or without modification,
are permitted in any medium without royalty provided the copyright
and if that doesn't work, install pre-built binaries of GCC for HP-UX.
+ HP-UX `make' updates targets which have the same time stamps as
+their prerequisites, which makes it generally unusable when shipped
+generated files such as `configure' are involved. Use GNU `make'
+instead.
+
On OSF/1 a.k.a. Tru64, some versions of the default C compiler cannot
parse its `<wchar.h>' header file. The option `-nodtk' can be used as
a workaround. If GNU CC is not installed, it is therefore recommended
## run aclocal and automake.
SUBDIRS = lib . doc m4 tests
+TEST_SUBDIRS = lib/Automake/tests tests
+
bin_SCRIPTS = automake aclocal
CLEANFILES = $(bin_SCRIPTS)
$(AM_V_GEN)cp $(srcdir)/lib/INSTALL $@
## recheck: convenience proxy target for the test suites.
-TEST_SUBDIRS = lib/Automake/tests tests
-
.PHONY: recheck
recheck:
@failcom='exit 1'; \
sc_tests_plain_aclocal \
sc_tests_plain_perl \
sc_tests_required_after_defs \
+sc_tests_tap_plan \
sc_tests_overriding_macros_on_cmdline \
sc_tests_plain_sleep \
sc_tests_plain_egrep_fgrep \
.PHONY: $(syntax_check_rules)
$(syntax_check_rules): automake aclocal
-maintainer-check: $(syntax_check_rules)
+
+## Check that the list of tests given in the Makefile is equal to the
+## list of all test scripts in the Automake testsuite.
+.PHONY: maintainer-check-list-of-tests
+maintainer-check-list-of-tests:
+ @failcom='exit 1'; \
+ for f in x $$MAKEFLAGS; do \
+ case $$f in \
+ *=* | --[!k]*);; \
+ *k*) failcom='fail=yes';; \
+ esac; \
+ done; \
+ for subdir in $(TEST_SUBDIRS); do \
+ (cd $$subdir && $(MAKE) $(AM_MAKEFLAGS) $@) || eval $$failcom; \
+ done; \
+ test -z "$$fail"
+
+maintainer-check: $(syntax_check_rules) maintainer-check-list-of-tests
## Look for test whose names can cause spurious failures when used as
## first argument to AC_INIT (chiefly because they might contain an
fi; \
done
+## TAP-based test scripts should not forget to declare a TAP plan. In
+## case it is not known in advance how many tests will be run, a "lazy"
+## plan can be used; but its use should be deliberate, explicitly declared
+## with a "plan_ later" call, rather than the result of an oversight.
+## This check helps to ensure this is indeed the case.
+sc_tests_tap_plan:
+ @with_plan=`grep -l '^ *plan_ ' $(srcdir)/tests/*.tap`; \
+ with_plan=`echo $$with_plan`; \
+ ok=:; \
+ for t in $(srcdir)/tests/*.tap; do \
+ case " $$with_plan " in \
+ *" $$t "*) ;; \
+ *) ok=false; echo $$t;; \
+ esac; \
+ done; \
+ $$ok || { \
+ echo 'The tests above do not declare a TAP plan.' 1>&2; \
+ exit 1; \
+ }
+
## Overriding a Makefile macro on the command line is not portable when
## recursive targets are used. Better use an envvar. SHELL is an
## exception, POSIX says it can't come from the environment. V, DESTDIR,
PRE_UNINSTALL = :
POST_UNINSTALL = :
build_triplet = @build@
+host_triplet = @host@
subdir = .
DIST_COMMON = README $(am__configure_deps) $(srcdir)/Makefile.am \
$(srcdir)/Makefile.in $(top_srcdir)/configure AUTHORS COPYING \
docdir = @docdir@
dvidir = @dvidir@
exec_prefix = @exec_prefix@
+host = @host@
host_alias = @host_alias@
+host_cpu = @host_cpu@
+host_os = @host_os@
+host_vendor = @host_vendor@
htmldir = @htmldir@
includedir = @includedir@
infodir = @infodir@
top_builddir = @top_builddir@
top_srcdir = @top_srcdir@
SUBDIRS = lib . doc m4 tests
+TEST_SUBDIRS = lib/Automake/tests tests
bin_SCRIPTS = automake aclocal
CLEANFILES = $(bin_SCRIPTS)
AUTOMAKESOURCES = automake.in aclocal.in
-e 's,[@]configure_input[@],Generated from $@.in; do not edit by hand.,g' \
-e 's,[@]datadir[@],$(datadir),g'
-TEST_SUBDIRS = lib/Automake/tests tests
# Perl coverage statistics.
PERL_COVERAGE_DB = $(abs_top_builddir)/cover_db
sc_tests_plain_aclocal \
sc_tests_plain_perl \
sc_tests_required_after_defs \
+sc_tests_tap_plan \
sc_tests_overriding_macros_on_cmdline \
sc_tests_plain_sleep \
sc_tests_plain_egrep_fgrep \
.PHONY: $(syntax_check_rules)
$(syntax_check_rules): automake aclocal
-maintainer-check: $(syntax_check_rules)
+
+.PHONY: maintainer-check-list-of-tests
+maintainer-check-list-of-tests:
+ @failcom='exit 1'; \
+ for f in x $$MAKEFLAGS; do \
+ case $$f in \
+ *=* | --[!k]*);; \
+ *k*) failcom='fail=yes';; \
+ esac; \
+ done; \
+ for subdir in $(TEST_SUBDIRS); do \
+ (cd $$subdir && $(MAKE) $(AM_MAKEFLAGS) $@) || eval $$failcom; \
+ done; \
+ test -z "$$fail"
+
+maintainer-check: $(syntax_check_rules) maintainer-check-list-of-tests
sc_test_names:
@m4_builtin_rx=`echo $(m4_builtins) | sed 's/ /|/g'`; \
m4_macro_rx="\\<($$m4_builtin_rx)\\>|\\<_?(A[CUMHS]|m4)_"; \
fi; \
done
+sc_tests_tap_plan:
+ @with_plan=`grep -l '^ *plan_ ' $(srcdir)/tests/*.tap`; \
+ with_plan=`echo $$with_plan`; \
+ ok=:; \
+ for t in $(srcdir)/tests/*.tap; do \
+ case " $$with_plan " in \
+ *" $$t "*) ;; \
+ *) ok=false; echo $$t;; \
+ esac; \
+ done; \
+ $$ok || { \
+ echo 'The tests above do not declare a TAP plan.' 1>&2; \
+ exit 1; \
+ }
+
sc_tests_overriding_macros_on_cmdline:
@if grep -E '\$$MAKE .*(SHELL=.*=|=.*SHELL=)' $(srcdir)/tests/*.test; then \
echo 'Rewrite "$$MAKE foo=bar SHELL=$$SHELL" as "foo=bar $$MAKE -e SHELL=$$SHELL"' 1>&2; \
- New `cscope' target to build a cscope database for the source tree.
+* Changes to Automake-generated testsuite harnesses:
+
+ - Test scripts that exit with status 99 to signal an "hard error" (e.g.,
+ and unexpected or internal error, or a failure to set up the test case
+ scenario) have their outcome reported as an 'ERROR' now. Previous
+ versions of automake reported such an outcome as a 'FAIL' (the only
+ difference with normal failures being that hard errors were counted
+ as failures even when the test originating them was listed in
+ XFAIL_TESTS).
+
+ - The testsuite summary displayed by the parallel-test harness has a
+ completely new format, that always list the numbers of passed, failed,
+ xfailed, xpassed, skipped and errored tests, even when these numbers
+ are zero (but using smart coloring when the color-tests option is in
+ effect).
+
+ - The default testsuite driver offered by the 'parallel-tests' option is
+ now implemented (partly at least) with the help of automake-provided
+ auxiliary scripts (e.g., `test-driver'), instead of relying entirely
+ on code in the generated Makefile.in.
+ This has two noteworthy implications. The first one is that projects
+ using the `parallel-tests' option should now either run automake with
+ the `--add-missing' option, or manually copy the `test-driver' script
+ into their tree. The second, and more important, implication is that
+ now, when the `parallel-tests' option is in use, TESTS_ENVIRONMENT can
+ not be used anymore to define a test runner, and the command specified
+ in LOG_COMPILER (and <ext>_LOG_COMPILER) must be a *real* executable
+ program or script. For example, this is still a valid usage (albeit
+ a little contorted):
+
+ TESTS_ENVIRONMENT = \
+ if test -n '$(STRICT_TESTS)'; then \
+ maybe_errexit='-e'; \
+ else \
+ maybe_errexit=''; \
+ fi;
+ LOG_COMPILER = $(SHELL) $$maybe_errexit
+
+ while this is not anymore:
+
+ TESTS_ENVIRONMENT = \
+ $(SHELL) `test -n '$(STRICT_TESTS_CHECKING)' && echo ' -e'`
+
+ neither is this:
+
+ TESTS_ENVIRONMENT = \
+ run_with_perl_or_shell () \
+ { \
+ if grep -q '^#!.*perl' $$1; then
+ $(PERL) $$1; \
+ else \
+ $(SHELL) $$1; \
+ fi; \
+ }
+ LOG_COMPILER = run_with_per_or_shell
+
+ - The package authors can now use customary testsuite drivers within
+ the framework provided by the 'parallel-tests' testsuite harness.
+ Consistently with the existing syntax, this can be done by defining
+ special makefile variables `LOG_DRIVER' and `<ext>_LOG_DRIVER'.
+
+ - A new developer-reserved variable `AM_TESTS_FD_REDIRECT' can be used
+ to redirect/define file descriptors used by the test scripts.
+
+ - The parallel-tests harness generates now, in addition the `.log' files
+ holding the output produced by the test scripts, a new set of `.trs'
+ files, holding "metadata" derived by the execution of the test scripts;
+ among such metadata are the outcomes of the test cases run by a script.
+
+ - Initial and still experimental support for the TAP test protocol is
+ now provided.
+
* Miscellaneous changes:
- The `dist' and `dist-all' targets now can run compressors in parallel.
Roman Fietze roman.fietze@telemotive.de
Ronald Landheer ronald@landheer.com
Roumen Petrov bugtrack@roumenpetrov.info
+Russ Allbery rra@stanford.edu
Rusty Ballinger rusty@rlyeh.engr.sgi.com
Ryan T. Sammartino ryants@shaw.ca
Sam Hocevar sam@zoy.org
append_exeext { exists $known_programs{$_[0]} } 'XFAIL_TESTS'
if (var ('XFAIL_TESTS'));
- if (option 'parallel-tests')
+ if (my $parallel_tests = option 'parallel-tests')
{
define_variable ('TEST_SUITE_LOG', 'test-suite.log', INTERNAL);
define_variable ('TEST_SUITE_HTML', '$(TEST_SUITE_LOG:.log=.html)', INTERNAL);
return substr ($obj, 0, length ($obj) - length ($test_suffix)) . '.log'
if substr ($obj, - length ($test_suffix)) eq $test_suffix;
}
+ my $base = $obj;
$obj .= '.log';
+ # The "test driver" program, deputed to handle tests protocol used by
+ # test scripts. By default, it's assumed that no protocol is used,
+ # so we fall back to the old "parallel-tests" behaviour, implemented
+ # by the `test-driver' auxiliary script.
+ if (! var 'LOG_DRIVER')
+ {
+ require_conf_file ($parallel_tests->{position}, FOREIGN,
+ 'test-driver');
+ define_variable ('LOG_DRIVER',
+ "\$(SHELL) $am_config_aux_dir/test-driver",
+ INTERNAL);
+ }
+ my $driver = '$(LOG_DRIVER)';
+ my $driver_flags = '$(AM_LOG_DRIVER_FLAGS) $(LOG_DRIVER_FLAGS)';
my $compile = 'LOG_COMPILE';
define_variable ($compile,
'$(LOG_COMPILER) $(AM_LOG_FLAGS) $(LOG_FLAGS)', INTERNAL);
$output_rules .= file_contents ('check2', new Automake::Location,
GENERIC => 0,
OBJ => $obj,
+ BASE => $base,
SOURCE => $val,
+ DRIVER => $driver,
+ DRIVER_FLAGS => $driver_flags,
COMPILE =>'$(' . $compile . ')',
EXT => '',
am__EXEEXT => 'FALSE');
{
(my $ext = $test_suffix) =~ s/^\.//;
$ext = uc $ext;
+ # See comments about definition of LOG_DRIVER, above.
+ if (! var "${ext}_LOG_DRIVER")
+ {
+ require_conf_file ($parallel_tests->{position}, FOREIGN,
+ 'test-driver');
+ define_variable ("${ext}_LOG_DRIVER",
+ "\$(SHELL) $am_config_aux_dir/test-driver",
+ INTERNAL);
+ }
+ my $driver = '$(' . $ext . '_LOG_DRIVER)';
+ my $driver_flags = '$(AM_' . $ext . '_LOG_DRIVER_FLAGS) ' .
+ '$(' . $ext . '_LOG_DRIVER_FLAGS)';
my $compile = $ext . '_LOG_COMPILE';
define_variable ($compile,
'$(' . $ext . '_LOG_COMPILER) $(AM_' . $ext . '_LOG_FLAGS)'
$output_rules .= file_contents ('check2', new Automake::Location,
GENERIC => 1,
OBJ => '',
+ BASE => '$*',
SOURCE => '$<',
+ DRIVER => $driver,
+ DRIVER_FLAGS => $driver_flags,
COMPILE => '$(' . $compile . ')',
EXT => $test_suffix,
am__EXEEXT => $am_exeext);
}
}
-
- define_variable ('TEST_LOGS_TMP', '$(TEST_LOGS:.log=.log-t)', INTERNAL);
-
- $clean_files{'$(TEST_LOGS_TMP)'} = MOSTLY_CLEAN;
$clean_files{'$(TEST_LOGS)'} = MOSTLY_CLEAN;
+ $clean_files{'$(TEST_LOGS:.log=.trs)'} = MOSTLY_CLEAN;
$clean_files{'$(TEST_SUITE_LOG)'} = MOSTLY_CLEAN;
$clean_files{'$(TEST_SUITE_HTML)'} = MOSTLY_CLEAN;
}
# Create temporary replacement for automake.
dosubst automake.in automake.tmp
-# Create required makefile snippets.
+# Create required makefile snippet.
cd tests
-$BOOTSTRAP_SHELL ./gen-parallel-tests > parallel-tests.am
-$BOOTSTRAP_SHELL ./instspc-tests.sh --generate-makefile > instspc-tests.am
+$BOOTSTRAP_SHELL ./gen-wrap-tests > wrap-tests.tmp
+chmod a-w wrap-tests.tmp
+mv -f wrap-tests.tmp wrap-tests.am
cd ..
# Run the autotools.
am_AUTORECONF
am_AUTOM4TE
am_AUTOCONF
+host_os
+host_vendor
+host_cpu
+host
build_os
build_vendor
build_cpu
System types:
--build=BUILD configure for building on BUILD [guessed]
+ --host=HOST cross-compile to build programs to run on HOST [BUILD]
_ACEOF
fi
case $build_os in *\ *) build_os=`echo "$build_os" | sed 's/ /-/g'`;; esac
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking host system type" >&5
+$as_echo_n "checking host system type... " >&6; }
+if ${ac_cv_host+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ if test "x$host_alias" = x; then
+ ac_cv_host=$ac_cv_build
+else
+ ac_cv_host=`$SHELL "$ac_aux_dir/config.sub" $host_alias` ||
+ as_fn_error $? "$SHELL $ac_aux_dir/config.sub $host_alias failed" "$LINENO" 5
+fi
+
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_host" >&5
+$as_echo "$ac_cv_host" >&6; }
+case $ac_cv_host in
+*-*-*) ;;
+*) as_fn_error $? "invalid value of canonical host" "$LINENO" 5;;
+esac
+host=$ac_cv_host
+ac_save_IFS=$IFS; IFS='-'
+set x $ac_cv_host
+shift
+host_cpu=$1
+host_vendor=$2
+shift; shift
+# Remember, the first character of IFS is used to create $*,
+# except with old shells:
+host_os=$*
+IFS=$ac_save_IFS
+case $host_os in *\ *) host_os=`echo "$host_os" | sed 's/ /-/g'`;; esac
+
+
+
# Save the AUTOCONF setting before AM_INIT_AUTOMAKE overrides it; this
# way we can run Autoconf tests from configure (or from the test
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether /bin/sh has working 'set -e' with exit trap" >&5
-$as_echo_n "checking whether /bin/sh has working 'set -e' with exit trap... " >&6; }
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether $SHELL has working 'set -e' with exit trap" >&5
+$as_echo_n "checking whether $SHELL has working 'set -e' with exit trap... " >&6; }
if ${am_cv_sh_errexit_works+:} false; then :
$as_echo_n "(cached) " >&6
else
- if /bin/sh -ec "trap 'exit \$?' 0; (exit 77); exit 77"; test $? = 77
+ if $SHELL -ec "trap 'exit \$?' 0; (exit 77); exit 77"; test $? = 77
then
am_cv_sh_errexit_works=yes
else
# Process this file with autoconf to produce a configure script.
# Copyright (C) 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003,
-# 2004, 2006, 2007, 2008, 2009, 2010 Free Software Foundation, Inc.
+# 2004, 2006, 2007, 2008, 2009, 2010, 2011 Free Software Foundation,
+# Inc.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
AC_CONFIG_SRCDIR(automake.in)
AC_CONFIG_AUX_DIR(lib)
+AC_CANONICAL_HOST
AC_CANONICAL_BUILD
# Save the AUTOCONF setting before AM_INIT_AUTOMAKE overrides it; this
AC_PROG_EGREP
AC_PROG_FGREP
-AC_CACHE_CHECK([whether /bin/sh has working 'set -e' with exit trap],
+AC_CACHE_CHECK([whether $SHELL has working 'set -e' with exit trap],
[am_cv_sh_errexit_works],
-[if /bin/sh -ec "trap 'exit \$?' 0; (exit 77); exit 77"; test $? = 77
+[if $SHELL -ec "trap 'exit \$?' 0; (exit 77); exit 77"; test $? = 77
then
am_cv_sh_errexit_works=yes
else
PRE_UNINSTALL = :
POST_UNINSTALL = :
build_triplet = @build@
+host_triplet = @host@
subdir = doc
DIST_COMMON = $(automake_TEXINFOS) $(dist_doc_DATA) $(dist_man1_MANS) \
$(dist_noinst_DATA) $(srcdir)/Makefile.am \
docdir = @docdir@
dvidir = @dvidir@
exec_prefix = @exec_prefix@
+host = @host@
host_alias = @host_alias@
+host_cpu = @host_cpu@
+host_os = @host_os@
+host_vendor = @host_vendor@
htmldir = @htmldir@
includedir = @includedir@
infodir = @infodir@
Support for test suites
-* Simple Tests:: Listing programs and scripts in @code{TESTS}
-* Simple Tests using parallel-tests:: More powerful test driver
-* DejaGnu Tests:: Interfacing with the external testing framework
+* Generalities about Testing:: Generic concepts and terminology about testing
+* Simple Tests:: Listing test scripts in @code{TESTS}
+* Custom Test Drivers:: Writing and using custom test drivers
+* Using the TAP test protocol:: Integrating test scripts that use the TAP protocol
+* DejaGnu Tests:: Interfacing with the @command{dejagnu} testing framework
* Install Tests:: Running tests on installed packages
+Simple Tests
+
+* Scripts-based Testsuites:: Automake-specific concepts and terminology
+* Serial Test Harness:: Older (and obsolescent) serial test harness
+* Parallel Test Harness:: Generic concurrent test harness
+
+Using the TAP test protocol
+
+* Introduction to TAP::
+* Use TAP with the Automake test harness::
+* Incompatibilities with other TAP parsers and drivers::
+* Links and external resources on TAP::
+
+Custom Test Drivers
+
+* Overview of Custom Test Drivers Support::
+* Declaring Custom Test Drivers::
+* API for Custom Test Drivers::
+
+API for Custom Test Drivers
+
+* Command-line arguments for test drivers::
+* Log files generation and test results recording::
+* Testsuite progress output::
+* HTML generation from testsuite logs::
+
Changing Automake's Behavior
* Options generalities:: Semantics of Automake option
multilibs (@pxref{Multilibs}). This file is maintained in the GCC
tree at @url{http://gcc.gnu.org/svn.html}.
+@item test-driver
+This implements the default test driver offered by the parallel
+testsuite harness.
+
@item texinfo.tex
Not a program, this file is required for @samp{make dvi}, @samp{make
ps} and @samp{make pdf} to work when Texinfo sources are in the
@item AC_CONFIG_AUX_DIR
Automake will look for various helper scripts, such as
@file{install-sh}, in the directory named in this macro invocation.
-@c This list is accurate relative to version 1.8
-(The full list of scripts is: @file{ar-lib}, @file{config.guess},
-@file{config.sub}, @file{depcomp}, @file{elisp-comp}, @file{compile},
-@file{install-sh}, @file{ltmain.sh}, @file{mdate-sh}, @file{missing},
-@file{mkinstalldirs}, @file{py-compile}, @file{texinfo.tex}, and
-@file{ylwrap}.) Not all scripts are always searched for; some scripts
+@c This list is accurate relative to version 1.11
+(The full list of scripts is:
+@file{ar-lib},
+@file{config.guess},
+@file{config.sub},
+@file{depcomp},
+@file{elisp-comp},
+@file{compile},
+@file{install-sh},
+@file{ltmain.sh},
+@file{mdate-sh},
+@file{missing},
+@file{mkinstalldirs},
+@file{py-compile},
+@file{test-driver},
+@file{texinfo.tex},
+@file{ylwrap}.)
+Not all scripts are always searched for; some scripts
will only be sought if the generated @file{Makefile.in} requires them.
If @code{AC_CONFIG_AUX_DIR} is not given, the scripts are looked for in
@cindex @code{make check}
@trindex check
-Automake supports three forms of test suites, the first two of which
-are very similar.
+Automake can generate code to handle two kinds of test suites. One is
+based on integration with the @command{dejagnu} framework. The other
+(and most used) form is based on the use of generic test scripts, and
+its activation is triggered by the definition of the special @code{TESTS}
+variable. This second form allows for various degrees of sophistication
+and customization; in particular, it allows for concurrent execution
+of test scripts, use of established test protocols such as TAP, and
+definition of custom test drivers and test runners.
+
+@noindent
+In either case, the testsuite is invoked via @samp{make check}.
@menu
-* Simple Tests:: Listing programs and scripts in @code{TESTS}
-* Simple Tests using parallel-tests:: More powerful test driver
-* DejaGnu Tests:: Interfacing with the external testing framework
+* Generalities about Testing:: Concepts and terminology about testing
+* Simple Tests:: Listing test scripts in @code{TESTS}
+* Custom Test Drivers:: Writing and using custom test drivers
+* Using the TAP test protocol:: Integrating test scripts that use the TAP protocol
+* DejaGnu Tests:: Interfacing with the @command{dejagnu} testing framework
* Install Tests:: Running tests on installed packages
@end menu
+@node Generalities about Testing
+@section Generalities about Testing
+
+The purpose of testing is to determine whether a program or system behaves
+as expected (e.g., known inputs produce the expected outputs, error
+conditions are correctly handled or reported, and older bugs do not
+resurface).
+
+@cindex test case
+The minimal unit of testing is usually called @emph{test case}, or simply
+@emph{test}. How a test case is defined or delimited, and even what
+exactly @emph{constitutes} a test case, depends heavily on the testing
+paradigm and/or framework in use, so we won't attempt any more precise
+definition. The set of the test cases for a given program or system
+constitutes its @emph{testsuite}.
+
+@cindex test harness
+@cindex testsuite harness
+A @emph{test harness} (also @emph{testsuite harness}) is a program or
+software component that executes all (or part of) the defined test cases,
+analyzes their outcomes, and report or register these outcomes
+appropriately. Again, the details of how this is accomplished (and how
+the developer and user can influence it or interface with it) varies
+wildly, and we'll attempt no precise definition.
+
+@cindex test pass
+@cindex test failure
+A test is said to @emph{pass} when it can determine that the condition or
+behaviour it means to verify holds, and is said to @emph{fail} when it can
+determine that such condition of behaviour does @emph{not} hold.
+
+@cindex test skip
+Sometimes, tests can rely on non-portable tools or prerequisites, or
+simply make no sense on a given system (for example, a test checking a
+Windows-specific feature makes no sense on a GNU/Linux system). In this
+case, accordingly to the definition above, the tests can neither be
+considered passed nor failed; instead, they are @emph{skipped} -- i.e.,
+they are not run, or their result is anyway ignored for what concerns
+the count of failures an successes. Skips are usually explicitly
+reported though, so that the user will be aware that not all of the
+testsuite has really run.
+
+@cindex xfail
+@cindex expected failure
+@cindex expected test failure
+@cindex xpass
+@cindex unexpected pass
+@cindex unexpected test pass
+It's not uncommon, especially during early development stages, that some
+tests fail for known reasons, and that the developer doesn't want to
+tackle these failures immediately (this is especially true when the
+failing tests deal with corner cases). In this situation, the better
+policy is to declare that each of those failures is an @emph{expected
+failure} (or @emph{xfail}). In case a test that is expected to fail ends
+up passing instead, many testing environments will flag the result as a
+special kind of failure called @emph{unexpected pass} (or @emph{xpass}).
+
+@cindex hard error
+@cindex Distinction between errors and failures in testsuites
+Many testing environments and frameworks distinguish between test failures
+and hard errors. As we've seen, a test failure happens when some invariant
+or expected behaviour of the software under test is not met. An @emph{hard
+error} happens when e.g., the set-up of a test case scenario fails, or when
+some other unexpected or highly undesirable condition is encountered (for
+example, the program under test experiences a segmentation fault).
+
+@emph{TODO}: Links to other test harnesses (esp. those sharing our
+terminology)?
+
@node Simple Tests
@section Simple Tests
-If the variable @code{TESTS} is defined, its value is taken to be a
-list of programs or scripts to run in order to do the testing.
-Programs needing data files should look for them in @code{srcdir}
-(which is both an environment variable and a make variable) so they
-work when building in a separate directory (@pxref{Build Directories,
-, Build Directories , autoconf, The Autoconf Manual}), and in
-particular for the @code{distcheck} rule (@pxref{Checking the
-Distribution}).
-
-For each of the @code{TESTS}, the result of execution is printed along
-with the test name, where @code{PASS} denotes a successful test,
-@code{FAIL} denotes a failed test, @code{XFAIL} an expected failure,
-@code{XPASS} an unexpected pass for a test that is supposed to fail,
-and @code{SKIP} denotes a skipped test.
+@menu
+* Scripts-based Testsuites:: Automake-specific concepts and terminology
+* Serial Test Harness:: Older (and obsolescent) serial test harness
+* Parallel Test Harness:: Generic concurrent test harness
+@end menu
+
+@node Scripts-based Testsuites
+@subsection Scripts-based Testsuites
+
+If the special variable @code{TESTS} is defined, its value is taken to be
+a list of programs or scripts to run in order to do the testing. Under
+the appropriate circumstances, it's possible for @code{TESTS} to list
+also data files to be passed to one or more test scripts defined by
+different means (the so-called ``log compilers'', @pxref{Parallel Test
+Harness}).
+
+Test scripts can be executed serially or concurrently. Automake
+supports both these kinds of test execution, with the serial test harness
+being the default (for backward-compatibility reasons only, as its use
+is nowadays discouraged). The concurrent test harness relies on the
+concurrence capabilities (if any) offered by the underlying @command{make}
+implementation, and can thus only be as good as those are.
+
+By default, only the exit statuses of the test scripts are considered when
+determining the testsuite outcome. But Automake allows also the use of
+more complex test protocols, either standard (@pxref{Using the TAP test
+protocol}) or custom (@pxref{Custom Test Drivers}). Note that you can
+enable such protocols only when the parallel harness is used: they won't
+work with the serial test harness. In the rest of this section we are
+going to concentrate mostly on protocol-less tests, since we'll have later
+a whole section devoted to the use of test protocols (again, @pxref{Custom
+Test Drivers}).
@cindex Exit status 77, special interpretation
+@cindex Exit status 99, special interpretation
+When no test protocol is in use, an exit status of 0 from a test script will
+denote a success, an exit status of 77 a skipped test, an exit status of 99
+an hard error, and any other exit status will denote a failure.
-The number of failures will be printed at the end of the run. If a
-given test program exits with a status of 77, then its result is ignored
-in the final count. This feature allows non-portable tests to be
-ignored in environments where they don't make sense.
+@cindex Tests, expected failure
+@cindex Expected test failure
+@vindex XFAIL_TESTS
+@vindex DISABLE_HARD_ERRORS
+@cindex Disabling hard errors
+You may define the variable @code{XFAIL_TESTS} to a list of tests
+(usually a subset of @code{TESTS}) that are expected to fail; this will
+effectively reverse the result of those tests (with the provision that
+skips and hard errors remain untouched). You may also instruct the
+testsuite harness to treat hard errors like simple failures, by defining
+the @code{DISABLE_HARD_ERRORS} make variable to a nonempty value.
+
+Note however that, for tests based on more complex test protocols,
+the exact effects of @code{XFAIL_TESTS} and @code{DISABLE_HARD_ERRORS}
+might change, or they might even have no effect at all (for example,
+@c Keep this in sync with tap-no-disable-hard-errors.test.
+in tests using TAP, there is not way to disable hard errors, and the
+@code{DISABLE_HARD_ERRORS} variable has no effect on them).
+
+@anchor{Testsuite progress on console}
+@cindex Testsuite progress on console
+The result of each test case run by the scripts in @code{TESTS} will be
+printed on standard output, along with the test name. For test protocols
+that allow more test cases per test script (such as TAP), a number,
+identifier and/or brief description specific for the single test case is
+expected to be printed in addition to the name of the test script. The
+possible results (whose meanings should be clear from the previous
+@ref{Generalities about Testing}) are @code{PASS}, @code{FAIL},
+@code{SKIP}, @code{XFAIL}, @code{XPASS} and @code{ERROR}. Here is an
+example of output from an hypothetical testsuite that uses both plain
+and TAP tests:
+@c Keep in sync with tap-doc.test.
+@example
+PASS: foo.sh
+PASS: zardoz.tap 1 - Daemon started
+PASS: zardoz.tap 2 - Daemon responding
+SKIP: zardoz.tap 3 - Daemon uses /proc # SKIP /proc is not mounted
+PASS: zardoz.tap 4 - Daemon stopped
+SKIP: bar.sh
+PASS: mu.tap 1
+XFAIL: mu.tap 2 # TODO frobnication not yet implemented
+@end example
+
+@noindent
+A testsuite summary (expected to report at least the number of run,
+skipped and failed tests) will be printed at the end of the testsuite
+run.
+@anchor{Simple tests and color-tests}
@vindex AM_COLOR_TESTS
+@cindex Colorized testsuite output
If the Automake option @code{color-tests} is used (@pxref{Options})
and standard output is connected to a capable terminal, then the test
results and the summary are colored appropriately. The user can disable
colored output by setting the @command{make} variable
@samp{AM_COLOR_TESTS=no}, or force colored output even without a connecting
-terminal with @samp{AM_COLOR_TESTS=always}.
-
-Note that the semantics of some @command{make} implementations when used
-in parallel mode (@pxref{Parallel make,,, autoconf, The Autoconf Manual})
-can cause the automatic detection of a connection to a capable terminal
-to fail. In that case, you can still resort to the use of
-@samp{AM_COLOR_TESTS=always}.
+terminal with @samp{AM_COLOR_TESTS=always}. It's also worth noting that
+some @command{make} implementations, when used in parallel mode, have
+slightly different semantics (@pxref{Parallel make,,, autoconf,
+The Autoconf Manual}), which can break the automatic detection of a
+connection to a capable terminal. If this is the case, you'll have to
+resort to the use of @samp{AM_COLOR_TESTS=always} in order to have the
+testsuite output colorized.
+
+Test programs that need data files should look for them in @code{srcdir}
+(which is both a make variable and an environment variable made available
+to the tests), so that they work when building in a separate directory
+(@pxref{Build Directories, , Build Directories , autoconf,
+The Autoconf Manual}), and in particular for the @code{distcheck} rule
+(@pxref{Checking the Distribution}).
@vindex TESTS
@vindex TESTS_ENVIRONMENT
-The variable @code{TESTS_ENVIRONMENT} can be used to set environment
-variables for the test run; the environment variable @env{srcdir} is
-set in the rule. If all your test programs are scripts, you can also
-set @code{TESTS_ENVIRONMENT} to an invocation of the shell (e.g.
-@samp{$(SHELL) -x} can be useful for debugging the tests), or any other
-interpreter. For instance, the following setup may be used to run tests
-with Perl:
+@vindex AM_TESTS_ENVIRONMENT
+The @code{AM_TESTS_ENVIRONMENT} and @code{TESTS_ENVIRONMENT} variables can
+be used to run initialization code and set environment variables for the
+test scripts. The former variable is developer-reserved, and can be
+defined in the @file{Makefile.am}, while the latter is reserved for the
+user, which can employ it to extend or override the settings in the
+former; for this to work portably, however, the contents of a non-empty
+@code{AM_TESTS_ENVIRONMENT} @emph{must} be terminated by a semicolon.
+
+@vindex AM_TESTS_FD_REDIRECT
+The @code{AM_TESTS_FD_REDIRECT} variable can be used to define file
+descriptor redirections for the test scripts. One might think that
+@code{AM_TESTS_ENVIRONMENT} could be used for this purpose, but experience
+has shown that doing so portably is practically impossible. The main
+hurdle is constituted by Korn shells, which usually set the close-on-exec
+flag on file descriptors opened with the @command{exec} builtin, thus
+rendering an idiom like @code{AM_TESTS_ENVIRONMENT = exec 9>&2;}
+ineffectual. This issue also affects some Bourne shells, such as the
+HP-UX's @command{/bin/sh},
+@c FIXME: should we offer a link to the relevant discussions on the
+@c bug-autoconf list?
@c Keep in sync with tests-environment-backcompat.test.
@example
-TESTS_ENVIRONMENT = $(PERL) -Mstrict -w
-TESTS = foo.pl bar.pl baz.pl
+AM_TESTS_ENVIRONMENT = \
+## Some environment initializations are kept in a separate shell
+## file `tests-env.sh', which can make it easier to also run tests
+## from the command line.
+ . $(srcdir)/tests-env.sh; \
+## On Solaris, prefer more POSIX-compliant versions of the standard
+## tools by default.
+ if test -d /usr/xpg4/bin; then \
+ PATH=/usr/xpg4/bin:$$PATH; export PATH; \
+ fi;
+@c $$ restore font-lock
+## With this, the test scripts will be able to print diagnostic
+## messages to the original standard error stream, even if the test
+## driver redirects the stderr of the test scripts to a log file
+## before executing them.
+AM_TESTS_FD_REDIRECT = 9>&2
@end example
-Note that the @option{parallel-tests} driver provides a more elegant
-way to achieve the same effect, freeing the @code{TESTS_ENVIRONMENT}
-variable for the user to override (@pxref{Simple Tests using
-parallel-tests}).
-
-
-@cindex Tests, expected failure
-@cindex Expected test failure
-
-@vindex XFAIL_TESTS
-You may define the variable @code{XFAIL_TESTS} to a list of tests
-(usually a subset of @code{TESTS}) that are expected to fail. This will
-reverse the result of those tests.
+@noindent
+Note however that @code{AM_TESTS_ENVIRONMENT} is, for historical and
+implementation reasons, @emph{not} supported by the serial harness
+(@pxref{Serial Test Harness}).
Automake ensures that each file listed in @code{TESTS} is built before
-any tests are run; you can list both source and derived programs (or
-scripts) in @code{TESTS}; the generated rule will look both in
-@code{srcdir} and @file{.}. For instance, you might want to run a C
-program as a test. To do this you would list its name in @code{TESTS}
-and also in @code{check_PROGRAMS}, and then specify it as you would
-any other program.
+it is run; you can list both source and derived programs (or scripts)
+in @code{TESTS}; the generated rule will look both in @code{srcdir} and
+@file{.}. For instance, you might want to run a C program as a test.
+To do this you would list its name in @code{TESTS} and also in
+@code{check_PROGRAMS}, and then specify it as you would any other
+program.
Programs listed in @code{check_PROGRAMS} (and @code{check_LIBRARIES},
@code{check_LTLIBRARIES}...) are only built during @code{make check},
by the tests, not the tests themselves. Of course you can set
@code{TESTS = $(check_PROGRAMS)} if all your programs are test cases.
+@node Serial Test Harness
+@subsection Serial Test Harness
+
+@emph{NOTE:} This harness, while still being the default one, is
+obsolescent, and kept mostly for backward-compatibility reasons.
+The user is advised to use the parallel test harness instead
+(@pxref{Parallel Test Harness}).
-@node Simple Tests using parallel-tests
-@section Simple Tests using @samp{parallel-tests}
+The serial harness operates by simply running the tests serially, one at
+the time, without any I/O redirection. It's up to the user to implement
+logging of tests' output, if that's requited or desired.
+@c TODO: give an example of how this can be done.
+
+For historical and implementation reasons, the @code{AM_TESTS_ENVIRONMENT}
+variable is @emph{not} supported by this harness (it will be silently
+ignored if defined); only @code{TESTS_ENVIRONMENT} is, and it is to be
+considered a developer-reserved variable. This is done so that, when
+using the serial harness, @code{TESTS_ENVIRONMENT} can be defined to an
+invocation of an interpreter through which the tests are to be run.
+For instance, the following setup may be used to run tests with Perl:
+
+@example
+TESTS_ENVIRONMENT = $(PERL) -Mstrict -w
+TESTS = foo.pl bar.pl baz.pl
+@end example
+
+@noindent
+It's important to note that the use of @code{TESTS_ENVIRONMENT} endorsed
+here would be @emph{invalid} with the parallel harness. That harness
+provides a more elegant way to achieve the same effect, with the further
+benefit of freeing the @code{TESTS_ENVIRONMENT} variable for the user
+(@pxref{Parallel Test Harness}).
+
+Another, less serious limit of the serial harness is that it doesn't
+really distinguish between simple failures and hard errors; this is
+due to historical reasons only, and might be fixed in future Automake
+versions.
+
+@node Parallel Test Harness
+@subsection Parallel Test Harness
@cindex @option{parallel-tests}, Using
-The option @option{parallel-tests} (@pxref{Options}) enables a test
-suite driver that is mostly compatible to the simple test driver described
-in the previous section, but provides a few more features and slightly different
-semantics. It features concurrent execution of tests with @code{make -j},
-allows to specify inter-test dependencies, lazy reruns of tests that
-have not completed in a prior run, summary and verbose output in
-@samp{RST} (reStructuredText) and @samp{HTML} format, and hard errors
-for exceptional failures. Similar to the simple test driver,
-@code{AM_COLOR_TESTS}, @code{XFAIL_TESTS}, and
-the @code{check_*} variables are honored, and the environment variable
-@env{srcdir} is set during test execution. Also, @code{TESTS_ENVIRONMENT}
-is still honored, but is complemented by a new developer-reserved variable
-@code{AM_TESTS_ENVIRONMENT} (described below).
-
-This test driver is still experimental and may undergo changes in order
-to satisfy additional portability requirements.
+The parallel (or concurrent) test harness is enabled by the Automake option
+@option{parallel-tests} . It features concurrent execution of tests with
+@code{make -j}, allows to specify inter-test dependencies, lazy reruns of
+tests that have not completed in a prior run, summary and verbose output in
+@samp{RST} (reStructuredText) and @samp{HTML} format.
+This harness is still somewhat experimental and may undergo changes in
+order to satisfy additional portability requirements.
+
+@anchor{Basics of test metadata}
@vindex TEST_SUITE_LOG
@vindex TESTS
-The driver operates by defining a set of @command{make} rules to create
-a summary log file, @code{TEST_SUITE_LOG}, which defaults to
-@file{test-suite.log} and requires a @file{.log} suffix. This file
-depends upon log files created for each single test program listed in
-@code{TESTS}, which in turn contain all output produced by the
-corresponding tests.
+@cindex @file{.log} files
+@cindex @file{.trs} files
+@cindex test metadata
+The parallel test harness operates by defining a set of @command{make}
+rules that run the test scripts listed in @code{TESTS}, and, for each
+such script, save its output in a corresponding @file{.log} file and
+its results (and other ``metadata'', @pxref{API for Custom Test Drivers})
+in a corresponding @file{.trs} (as in @b{T}est @b{R}e@b{S}ults) file.
+@c We choose the `.trs' extension also because, at the time of writing,
+@c it isn't already used for other significant purposes; see e.g.:
+@c - http://filext.com/file-extension/trs
+@c - http://www.file-extensions.org/search/?searchstring=trs
+The @file{.log} file will contain all the output emitted by the test on
+its standard output and its standard error. The @file{.trs} file will
+contain, among the other things, the results of the test cases run by
+the script.
+
+The parallel test harness will also create a summary log file,
+@code{TEST_SUITE_LOG}, which defaults to @file{test-suite.log} and requires
+a @file{.log} suffix. This file depends upon all the @file{.log} and
+@file{.trs} files created for the test scripts listed in @code{TESTS}.
@vindex VERBOSE
-As with the simple driver above, by default one status line is printed
+As with the serial harness above, by default one status line is printed
per completed test, and a short summary after the suite has completed.
However, standard output and standard error of the test are redirected
to a per-test log file, so that parallel execution does not produce
intermingled output. The output from failed tests is collected in the
@file{test-suite.log} file. If the variable @samp{VERBOSE} is set, this
-file is output after the summary. For best results, the tests should be
-verbose by default now.
+file is output after the summary.
+@c FIXME: we should be clearer about what we mean exactly here ...
+For best results, the tests should be verbose by default now.
@vindex TEST_EXTENSIONS
@vindex TEST_LOGS
-Each log file is created when the corresponding test has completed.
-The set of log files is listed in the read-only variable
-@code{TEST_LOGS}, and defaults to @code{TESTS}, with the executable
-extension if any (@pxref{EXEEXT}), as well as any suffix listed in
-@code{TEST_EXTENSIONS} removed, and @file{.log} appended.
+Each couple of @file{.log} and @file{.trs} files is created when the
+corresponding test has completed. The set of log files is listed in
+the read-only variable @code{TEST_LOGS}, and defaults to @code{TESTS},
+with the executable extension if any (@pxref{EXEEXT}), as well as any
+suffix listed in @code{TEST_EXTENSIONS} removed, and @file{.log} appended.
@code{TEST_EXTENSIONS} defaults to @file{.test}. Results are undefined
if a test file name ends in several concatenated suffixes.
@vindex AM_@var{ext}_LOG_FLAGS
@vindex AM_LOG_FLAGS
For tests that match an extension @code{.@var{ext}} listed in
-@code{TEST_EXTENSIONS}, you can provide a test driver using the variable
-@code{@var{ext}_LOG_COMPILER} (note the upper-case extension) and pass
-options in @code{AM_@var{ext}_LOG_FLAGS} and allow the user to pass
-options in @code{@var{ext}_LOG_FLAGS}. It will cause all tests with
-this extension to be called with this driver. For all tests without a
-registered extension, the variables @code{LOG_COMPILER},
+@code{TEST_EXTENSIONS}, you can provide a custom ``test runner'' using
+the variable @code{@var{ext}_LOG_COMPILER} (note the upper-case
+extension) and pass options in @code{AM_@var{ext}_LOG_FLAGS} and allow
+the user to pass options in @code{@var{ext}_LOG_FLAGS}. It will cause
+all tests with this extension to be called with this runner. For all
+tests without a registered extension, the variables @code{LOG_COMPILER},
@code{AM_LOG_FLAGS}, and @code{LOG_FLAGS} may be used. For example,
@c Keep in sync with parallel-tests-log-compiler-example.test.
@noindent
will invoke @samp{$(PERL) -w foo.pl}, @samp{$(PYTHON) -v bar.py},
and @samp{./wrapper-script -d baz} to produce @file{foo.log},
-@file{bar.log}, and @file{baz.log}, respectively. The
-@samp{TESTS_ENVIRONMENT} variable is still expanded before the driver,
-but should be reserved for the user.
+@file{bar.log}, and @file{baz.log}, respectively. The @file{foo.trs},
+@file{bar.trs} and @file{baz.trs} files will be automatically produced
+as a side-effect.
-@vindex AM_TESTS_ENVIRONMENT
-The @code{AM_TESTS_ENVIRONMENT} variable can be used to run initialization
-code and set environment variables for the tests' runs. The user can
-still employ the @code{TESTS_ENVIRONMENT} variable to override settings
-from @code{AM_TESTS_ENVIRONMENT}; for that to work portably, however,
-the contents of a non-empty @code{AM_TESTS_ENVIRONMENT} @emph{must} be
-terminated by a semicolon. Here is an example of a slightly elaborate
-definition:
+It's important to note that, differently from what we've seen for the
+serial test harness (@pxref{Parallel Test Harness}), the
+@code{AM_TESTS_ENVIRONMENT} and @code{TESTS_ENVIRONMENT} variables
+@emph{cannot} be use to define a custom test runner; the
+@code{LOG_COMPILER} and @code{LOG_FLAGS} (or their extension-specific
+counterparts) should be used instead:
@example
-AM_TESTS_ENVIRONMENT = \
-## Some environment initializations are kept in a separate shell file
-## `tests-env.sh', which can make it easier to also run tests from the
-## command line.
- . $(srcdir)/tests-env.sh; \
-## On Solaris, prefer more POSIX-compliant versions of the standard tools
-## by default.
- if test -d /usr/xpg4/bin; then \
- PATH=/usr/xpg4/bin:$$PATH; export PATH; \
- fi;
-@c $$ restore font-lock
+## This is WRONG!
+AM_TESTS_ENVIRONMENT = PERL5LIB='$(srcdir)/lib' $(PERL) -Mstrict -w
+@end example
+
+@example
+## Do this instead.
+AM_TESTS_ENVIRONMENT = PERL5LIB='$(srcdir)/lib'; export PERL5LIB;
+LOG_COMPILER = $(PERL)
+AM_LOG_FLAGS = -Mstrict -w
@end example
@trindex mostlyclean
@trindex check-html
@vindex RST2HTML
@vindex TEST_SUITE_HTML
+@noindent
With @code{make check-html}, the log files may be converted from RST
(reStructuredText, see @uref{http://docutils.sourceforge.net/@/rst.html})
to HTML using @samp{RST2HTML}, which defaults to @command{rst2html} or
set of converted log files. The log and HTML files are removed upon
@code{make mostlyclean}.
-@vindex DISABLE_HARD_ERRORS
-@cindex Exit status 99, special interpretation
-@cindex hard error
-Even in the presence of expected failures (see @code{XFAIL_TESTS}), there
-may be conditions under which a test outcome needs attention. For
-example, with test-driven development, you may write tests for features
-that you have not implemented yet, and thus mark these tests as expected
-to fail. However, you may still be interested in exceptional conditions,
-for example, tests that fail due to a segmentation violation or another
-error that is independent of the feature awaiting implementation.
-Tests can exit with an exit status of 99 to signal such a @emph{hard
-error}. Unless the variable @code{DISABLE_HARD_ERRORS} is set to a
-nonempty value, such tests will be counted as failed.
-
-By default, the test suite driver will run all tests, but there are
+By default, the test suite harness will run all tests, but there are
several ways to limit the set of tests that are run:
@itemize @bullet
@item
-You can set the @code{TESTS} variable, similarly to how you can with
-the simple test driver from the previous section. For example, you can
-use a command like this to run only a subset of the tests:
+You can set the @code{TESTS} variable. For example, you can use a
+command like this to run only a subset of the tests:
@example
env TESTS="foo.test bar.test" make -e check
@item
@vindex RECHECK_LOGS
@cindex lazy test execution
-By default, the test driver removes all old per-test log files before it
-starts running tests to regenerate them. The variable
-@code{RECHECK_LOGS} contains the set of log files which are removed.
-@code{RECHECK_LOGS} defaults to @code{TEST_LOGS}, which means all tests
-need to be rechecked. By overriding this variable, you can choose which
-tests need to be reconsidered. For example, you can lazily rerun only
-those tests which are outdated, i.e., older than their prerequisite test
-files, by setting this variable to the empty value:
+By default, the test harness removes all old per-test @file{.log} and
+@file{.trs} files before it starts running tests to regenerate them. The
+variable @code{RECHECK_LOGS} contains the set of @file{.log} (and, by
+implication, @file{.trs}) files which are removed. @code{RECHECK_LOGS}
+defaults to @code{TEST_LOGS}, which means all tests need to be rechecked.
+By overriding this variable, you can choose which tests need to be
+reconsidered. For example, you can lazily rerun only those tests which
+are outdated, i.e., older than their prerequisite test files, by setting
+this variable to the empty value:
@example
env RECHECK_LOGS= make -e check
You can ensure that all tests are rerun which have failed or passed
unexpectedly, by running @code{make recheck} in the test directory.
This convenience target will set @code{RECHECK_LOGS} appropriately
-before invoking the main test driver. The @code{recheck-html} target
-does the same as @code{recheck} but again converts the resulting log
-file in HTML format, like the @code{check-html} target.
+before invoking the main test harness. The @code{recheck-html} target
+does the same as @code{recheck} but again converts the resulting
+@file{.log} file in HTML format, like the @code{check-html} target.
@end itemize
+@noindent
In order to guarantee an ordering between tests even with @code{make
--j@var{N}}, dependencies between the corresponding log files may be
-specified through usual @command{make} dependencies. For example, the
-following snippet lets the test named @file{foo-execute.test} depend
+-j@var{N}}, dependencies between the corresponding @file{.log} files
+may be specified through usual @command{make} dependencies. For example,
+the following snippet lets the test named @file{foo-execute.test} depend
upon completion of the test @file{foo-compile.test}:
@example
concurrent execution.
@cindex Unit tests
+@c Keep in sync with 'parallel-tests-extra-programs.test'.
The combination of lazy test execution and correct dependencies between
tests and their sources may be exploited for efficient unit testing
during development. To further speed up the edit-compile-test cycle, it
In case of doubt you may want to require to use GNU @command{make},
or work around the issue with inference rules to generate the tests.
+@node Custom Test Drivers
+@section Custom Test Drivers
+
+@menu
+* Overview of Custom Test Drivers Support::
+* Declaring Custom Test Drivers::
+* API for Custom Test Drivers::
+@end menu
+
+@node Overview of Custom Test Drivers Support
+@subsection Overview of Custom Test Drivers Support
+
+Starting from Automake version 1.12, the parallel test harness allows
+the package authors to use third-party custom test drivers, in case the
+default ones are inadequate for their purposes, or do not support their
+testing protocol of choice.
+
+A custom test driver is expected to properly run the test programs passed
+to it (including the command-line arguments passed to those programs, if
+any), to analyze their execution and outcome, to create the @file{.log}
+and @file{.trs} files associated to these test runs, and to display the test
+results on the console. It is responsibility of the author of the test
+driver to ensure that it implements all the above steps meaningfully and
+correctly; Automake isn't and can't be of any help here. On the other
+hand, the Automake-provided code for testsuite summary generation offers
+support for test drivers allowing several test results per test script,
+if they take care to register such results properly (@pxref{Log files
+generation and test results recording}).
+
+The exact details of how test scripts' results are to be determined and
+analyzed is left to the individual drivers. Some drivers might only
+consider the test script exit status (this is done for example by the
+default test driver used by the parallel test harness, described
+in the previous section). Other drivers might implement more complex and
+advanced test protocols, which might require them to parse and interpreter
+the output emitted by the test script they're running (examples of such
+protocols are TAP and SubUnit).
+
+It's very important to note that, even when using custom test drivers,
+most of the infrastructure described in the previous section about the
+parallel harness remains in place; this includes:
+
+@itemize
+@item
+list of test scripts defined in @code{TESTS}, and overridable at
+runtime through the redefinition of @code{TESTS} or @code{TEST_LOGS};
+@item
+concurrency through the use of @command{make}'s option @option{-j};
+@item
+per-test @file{.log} and @file{.trs} files, and generation of a summary
+@file{.log} file from them;
+@item
+@code{recheck} target, @code{RECHECK_LOGS} variable, and lazy reruns
+of tests;
+@item
+inter-test dependencies;
+@item
+support for @code{check_*} variables (@code{check_PROGRAMS},
+@code{check_LIBRARIES}, ...);
+@item
+use of @code{VERBOSE} environment variable to get verbose output on
+testsuite failures;
+@item
+definition and honoring of @code{TESTS_ENVIRONMENT},
+@code{AM_TESTS_ENVIRONMENT} and @code{AM_TESTS_FD_REDIRECT}
+variables;
+@item
+definition of generic and extension-specific @code{LOG_COMPILER} and
+@code{LOG_FLAGS} variables.
+@end itemize
+
+@noindent
+On the other hand, the exact semantics of how (and if)
+@option{color-tests}, @code{XFAIL_TESTS}, and hard errors are supported
+and handled is left to the individual test drivers.
+
+@c TODO: We should really add a working example in the doc/ directory,
+@c TODO: and reference if from here.
+
+@node Declaring Custom Test Drivers
+@subsection Declaring Custom Test Drivers
+
+@vindex _LOG_DRIVER
+@vindex _LOG_DRIVER_FLAGS
+@vindex LOG_DRIVER
+@vindex LOG_DRIVER_FLAGS
+@vindex @var{ext}_LOG_DRIVER
+@vindex @var{ext}_LOG_DRIVER_FLAGS
+@vindex AM_@var{ext}_LOG_DRIVER_FLAGS
+@vindex AM_LOG_DRIVER_FLAGS
+Custom testsuite drivers are declared by defining the make variables
+@code{LOG_DRIVER} or @code{@var{ext}_LOG_DRIVER} (where @var{ext} must
+be declared in @code{TEST_EXTENSIONS}). They must be defined to
+programs or scripts that will be used to drive the execution, logging,
+and outcome report of the tests with corresponding extensions (or of
+those with no registered extension in the case of @code{LOG_DRIVER}).
+Clearly, multiple distinct test drivers can be declared in the same
+@file{Makefile.am}. Note moreover that the @code{LOG_DRIVER} variables
+are @emph{not} a substitute for the @code{LOG_COMPILER} variables: the
+two sets of variables can, and often do, usefully and legitimately
+coexist.
+
+@c TODO: We should really be able to point to a clarifying example here!
+
+The developer-reserved variable @code{AM_LOG_DRIVER_FLAGS} and the
+user-reserved variable @code{LOG_DRIVER_FLAGS} can be used to define
+flags that will be passed to each invocation of @code{LOG_DRIVER},
+with the user-defined flags obviously taking precedence over the
+developer-reserved ones. Similarly, for each extension @var{ext}
+declared in @code{TEST_EXTENSIONS}, flags listed in
+@code{AM_@var{ext}_LOG_DRIVER_FLAGS} and
+@code{@var{ext}_LOG_DRIVER_FLAGS} will be passed to
+invocations of @code{@var{ext}_LOG_DRIVER}.
+
+@node API for Custom Test Drivers
+@subsection API for Custom Test Drivers
+
+Note that @emph{the APIs described here are still somewhat experimental},
+and might undergo changes and tightenings in the future, to accommodate
+for new features or to satisfy additional portability requirements.
+
+The main characteristic of these APIs is that they are designed to share
+as much infrastructure, semantics, and implementation details as possible
+with the parallel test harness and its default driver.
+
+@menu
+* Command-line arguments for test drivers::
+* Log files generation and test results recording::
+* Testsuite progress output::
+* HTML generation from testsuite logs::
+@end menu
+
+@node Command-line arguments for test drivers
+@subsubsection Command-line arguments for test drivers
+
+A custom driver can rely on various command-line options and arguments
+being passed to it automatically by the Automake's @option{parallel-tests}
+harness. It is @emph{mandatory} that it understands all of them (even
+if the exact interpretation of the associated semantics can legitimately
+change between a test driver and another, and even be a no-op in some
+drivers).
+
+@noindent
+Here is the list of options:
+
+@table @option
+@item --test-name=@var{NAME}
+The name of the test, with VPATH prefix (if any) removed. This can have a
+suffix and a directory component (as in e.g., @file{sub/foo.test}), and is
+mostly meant to be used in console reports about testsuite advancements and
+results (@pxref{Testsuite progress output}).
+@item --log-file=@file{@var{PATH}.log}
+The @file{.log} file the test driver must create (@pxref{Basics of
+test metadata}). If it has a directory component (as in e.g.,
+@file{sub/foo.log}), the test harness will ensure that such directory
+exists @emph{before} the test driver is called.
+@item --trs-file=@file{@var{PATH}.trs}
+The @file{.trs} file the test driver must create (@pxref{Basics of
+test metadata}). If it has a directory component (as in e.g.,
+@file{sub/foo.trs}), the test harness will ensure that such directory
+exists @emph{before} the test driver is called.
+@item --color-tests=@{yes|no@}
+Whether the console output should be colorized or not (@pxref{Simple
+tests and color-tests}, to learn when this option gets activated and
+when it doesn't).
+@item --expect-failure=@{yes|no@}
+Whether the tested program is expected to fail.
+@item --enable-hard-errors=@{yes|no@}
+Whether ``hard errors'' in the tested program should be treated differently
+from normal failures or not (the default should be @code{yes}). The exact
+meaning of ``hard error'' is highly dependent from the test protocols or
+conventions in use.
+@item --
+Explicitly terminate the list of options.
+@end table
+
+@noindent
+The first non-option argument passed to the test driver is the program to
+be run, and all the following ones are command-line options and arguments
+for this program.
+
+Note that the exact semantics attached to the @option{--color-tests},
+@option{--expect-failure} and @option{--enable-hard-errors} options are
+left up to the individual test drivers. Still, having a behaviour
+compatible or at least similar to that provided by the default
+@option{parallel-tests} driver is advised, as that would offer a better
+consistency and a more pleasant user experience.
+
+@node Log files generation and test results recording
+@subsubsection Log files generation and test results recording
+
+The test driver must correctly generate the files specified by the
+@option{--log-file} and @option{--trs-file} option (even when the tested
+program fails or crashes).
+
+The @file{.log} file should ideally contain all the output produced by the
+tested program, plus optionally other information that might facilitate
+debugging or analysis of bug reports. Apart from that, its format is
+basically free; the only limitation being that it must parse validly as
+reStructuredText if the @file{.log} -> @file{.html} conversion is to be
+supported.
+
+The @file{.trs} file is used to register some metadata through the use
+of custom reStructuredText fields. This metadata is expected to be
+employed in various ways by the parallel test harness; for example, to
+count the test results when printing the testsuite summary, or to decide
+which tests to re-run upon @command{make reheck}. Unrecognized metadata
+in a @file{.trs} file is currently ignored by the harness, but this might
+change in the future. The list of currently recognized metadata follows.
+
+@table @code
+
+@item :test-result:
+@cindex Register test result
+@cindex Register test case result
+@cindex Test result, registering
+@cindex Test case result, registering
+@cindex @code{:test-result:}
+@cindex reStructuredText field, @code{:test-result:}
+The test driver must use this field to register the results of @emph{each}
+test case run by a test script file. Several @code{:test-result:} fields
+can be present in the same @file{.trs} file; this is done in order to
+support test protocols that allow a single test script to run more test
+cases.
+
+@c Keep this in sync with lib/am/check-am:$(TEST_SUITE_LOG).
+The only recognized test results are currently @code{PASS}, @code{XFAIL},
+@code{SKIP}, @code{FAIL}, @code{XPASS} and @code{ERROR}. These results,
+when declared with @code{:test-result:}, can be optionally followed by
+text holding the name and/or a brief description of the corresponding
+test; the @option{parallel-tests} harness will ignore such extra text when
+generating @file{test-suite.log} and preparing the testsuite summary.
+
+@c Keep in sync with 'test-metadata-recheck.test'.
+@item @code{:recheck:}
+@cindex :recheck:
+@cindex reStructuredText field, @code{:recheck:}
+If this field is present and defined to @code{no}, then the corresponding
+test script will @emph{not} be run upon a @command{make recheck}. What
+happens when two or more @code{:recheck:} fields are present in the same
+@file{.trs} file is undefined behaviour.
+
+@c Keep in sync with 'test-metadata-global-log.test'.
+@item @code{:copy-in-global-log:}
+@cindex :copy-in-global-log:
+@cindex reStructuredText field, @code{:copy-in-global-log:}
+If this field is present and defined to @code{no}, then the content
+of the @file{.log} file will @emph{not} be copied into the global
+@file{test-suite.log}. We allow to forsake such copying because, while
+it can be useful in debugging and analysis of bug report, it can also be
+just a waste of space in normal situations, e.g., when a test script is
+successful. What happens when two or more @code{:copy-in-global-log:}
+fields are present in the same @file{.trs} file is undefined behaviour.
+
+@c Keep in sync with 'test-metadata-global-result.test'.
+@item @code{:test-global-result:}
+@cindex :test-global-result:
+@cindex reStructuredText field, @code{:test-global-result:}
+This is used to declare the "global result" of the script. Currently,
+the value of this field is needed only to be reported (more or less
+verbatim) in the generated global log file @code{$(TEST_SUITE_LOG)},
+so it's quite free-form. For example, a test script which run 10 test
+cases, 6 of which pass and 4 of which are skipped, could reasonably have
+a @code{PASS/SKIP} value for this field, while a test script which run
+19 successful tests and one failed test could have an @code{ALMOST
+PASSED} value. What happens when two or more @code{:test-global-result:}
+fields are present in the same @file{.trs} file is undefined behaviour.
+@end table
+
+@noindent
+Let's see a small example. Assume a @file{.trs} file contains the
+following lines:
+
+@example
+:test-result: PASS server starts
+:global-log-copy: no
+:test-result: PASS HTTP/1.1 request
+:test-result: FAIL HTTP/1.0 request
+:recheck: yes
+:test-result: SKIP HTTPS request (TLS library wasn't available)
+:test-result: PASS server stops
+@end example
+
+@noindent
+Then the corresponding test script will be re-run by @command{make check},
+will contribute with @emph{five} test results to the testsuite summary
+(three of these tests being successful, one failed, and one skipped), and
+the content of the corresponding @file{.log} file will @emph{not} be
+copied in the global log file @file{test-suite.log}.
+
+@node Testsuite progress output
+@subsubsection Testsuite progress output
+
+A custom test driver also has the task of displaying, on the standard
+output, the test results as soon as they become available. Depending on
+the protocol in use, it can also display the reasons for failures and
+skips, and, more generally, any useful diagnostic output (but remember
+that each line on the screen is precious, so that cluttering the screen
+with overly verbose information is bad idea). The exact format of this
+progress output is left up to the test driver; in fact, a custom test
+driver might @emph{theoretically} even decide not to do any such report,
+leaving it all to the testsuite summary (that would be a very lousy idea,
+of course, and serves only to illustrate the flexibility that is
+granted here).
+
+Remember that consistency is good; so, if possible, try to be consistent
+with the output of the built-in Automake test drivers, providing a similar
+``look & feel''. In particular, the testsuite progress output should be
+colorized when the @option{--color-tests} is passed to the driver. On the
+other end, if you are using a known and widespread test protocol with
+well-established implementations, being consistent with those
+implementations' output might be a good idea too.
+
+@c TODO: Give an example, maybe inspired to py.test-style output.
+@c TODO: That is a good idea because it shows a test driver that allows
+@c TODO: for different levels of verbosity in the progress output (could
+@c TODO: be implemented either using a driver cmdline flag, or an
+@c TODO: environment variable, or both).
+
+@node HTML generation from testsuite logs
+@subsubsection HTML generation from testsuite logs
+
+If HTML testsuite output (with @code{check-html}) is to be supported,
+the generated @file{.log} files must contain syntactically valid
+reStructuredText. If this is not the case, the HTML generation will not
+work, although all the other functionalities of the Automake testsuite
+harness should remain untouched, and continue to work correctly.
+
+@node Using the TAP test protocol
+@section Using the TAP test protocol
+
+@menu
+* Introduction to TAP::
+* Use TAP with the Automake test harness::
+* Incompatibilities with other TAP parsers and drivers::
+* Links and external resources on TAP::
+@end menu
+
+@node Introduction to TAP
+@subsection Introduction to TAP
+
+TAP, the Test Anything Protocol, is a simple text-based interface between
+testing modules or programs and a test harness. The tests (also called
+``TAP producers'' in this context) write test results in a simple format
+on standard output; a test harness (also called ``TAP consumer'') will
+parse and interpret these results, and properly present them to the user,
+and/or register them for later analysis. The exact details of how this
+is accomplished can vary among different test harnesses. The Automake
+parallel harness will present the results on the console in the usual
+fashion (@pxref{Testsuite progress on console}), and will use the
+@file{.trs} files (@pxref{Basics of test metadata}) to store the test
+results and related metadata. Apart from that, it will try to remain
+as much compatible as possible with pre-existing and widespread utilities,
+such as the @uref{http://search.cpan.org/~andya/Test-Harness/bin/prove,
+@command{prove} utility}, at least for the simpler usages.
+
+TAP started its life as part of the test harness for Perl, but today
+it has been (mostly) standardized, and has various independent
+implementations in different languages; among them, C, C++, Perl,
+Python, PHP, and Java. For a semi-official specification of the
+TAP protocol, please refer to the documentation of
+@uref{http://search.cpan.org/~petdance/Test-Harness/lib/Test/Harness/TAP.pod,
+ @samp{Test::Harness::TAP}}.
+
+The most relevant real-world usages of TAP are obviously in the testsuites
+of @command{perl} and of many perl modules. Still, also other important
+third-party packages, such as @uref{http://git-scm.com/, @command{git}},
+use TAP in their testsuite.
+
+@node Use TAP with the Automake test harness
+@subsection Use TAP with the Automake test harness
+
+Currently, the TAP driver that comes with Automake requires a perl
+interpreter to work, and requires various by-hand steps on the
+developer's part (this should be fixed in future Automake versions).
+You'll have grab the @file{tap-driver.pl} script from the Automake
+distribution by hand, copy it in your source tree, add code to
+@file{configure.ac} to search a perl interpreter and to define the
+@code{$(PERL)} variable accordingly, and use the Automake support
+for third-party test drivers to instruct the harness to use the
+@file{tap-driver.pl} to run your TAP-producing tests. See the example
+below for clarification.
+
+Apart from the options common to all the Automake test drivers
+(@pxref{Command-line arguments for test drivers}), the @file{tap-driver.pl}
+supports the following options, whose names are chosen for enhanced
+compatibility with the @command{prove} utility.
+
+@table @option
+@c Keep in sync with 'tap-exit.test' and 'tap-signal.test'.
+@item --ignore-exit
+Causes the test driver to ignore the exit status of the test scripts;
+by default, the driver will report an error if the script exit with a
+non-zero status. This option has effect also
+@item --comments
+Instruct the test driver to display TAP diagnostic (i.e., lines beginning
+with the @samp{#} character) in the testsuite progress output too; by
+default, TAP diagnostic is only copied in the @file{.log} file.
+@item --no-comments
+Revert the effects of @option{--comments}.
+@item --merge
+Instruct the test driver to merge the test scripts' standard error into
+their standard output. This is necessary if you want to ensure that
+diagnostics from the test scripts are displayed in the correct order
+relative to test results; this can be of great help in debugging
+(especially if your test scripts are shell scripts run with shell
+tracing active). As a downside, this option might cause the test
+harness to get confused if anything that appears on standard error
+looks like a test result.
+@item --no-merge
+Revert the effects of @option{--merge}.
+@item --diagnostic-string=@var{STRING}
+Change the string that introduces TAP diagnostic from the default value
+of ``@code{#}'' to @code{@var{STRING}}. This can be useful if your
+TAP-based test scripts produce verbose output on which they have limited
+control (because, say, the output comes by other tools invoked in the
+scripts), and it might contain text that gets spuriously interpreted as
+TAP diagnostic: such an issue can be solved by redefining the string that
+activates TAP diagnostic to a value you know won't appear by chance in
+the tests' output. Note however that this feature is non-standard, as
+the ``official'' TAP protocol does not allow for such a customization; so
+don't use it if you can avoid it.
+@end table
+
+@noindent
+Here is an example of how the TAP driver can be set up and used.
+
+@c Keep in sync with tap-doc2.test.
+@example
+% @kbd{cat configure.ac}
+AC_INIT([GNU Try Tap], [1.0], [bug-automake@@gnu.org])
+AC_CONFIG_AUX_DIR([build-aux])
+AM_INIT_AUTOMAKE([foreign parallel-tests -Wall -Werror])
+AC_CONFIG_FILES([Makefile])
+AC_REQUIRE_AUX_FILE([tap-driver.pl])
+AC_PATH_PROG([PERL], [perl])
+test -n "$PERL" || AC_MSG_ERROR([perl not found])
+$PERL -MTAP::Parser -e 1 || AC_MSG_ERROR([TAP::Parser not found])
+AC_OUTPUT
+
+% @kbd{cat Makefile.am}
+TEST_LOG_DRIVER = $(PERL) $(srcdir)/build-aux/tap-driver.pl
+TESTS = foo.test bar.test baz.test
+EXTRA_DIST = $(TESTS)
+
+% @kbd{cat foo.test}
+#!/bin/sh
+echo 1..4 # Number of tests to be executed.
+echo 'ok 1 - Swallows fly'
+echo 'not ok 2 - Caterpillars fly # TODO metamorphosis in progress'
+echo 'ok 3 - Pigs fly # SKIP not enough acid'
+echo '# I just love word plays ...'
+echo 'ok 4 - Flies fly too :-)'
+
+% @kbd{cat bar.test}
+#!/bin/sh
+echo 1..3
+echo 'not ok 1 - Bummer, this test has failed.'
+echo 'ok 2 - This passed though.'
+echo 'Bail out! Ennui kicking in, sorry...'
+echo 'ok 3 - This will not be seen.'
+
+% @kbd{cat baz.test}
+#!/bin/sh
+echo 1..1
+echo ok 1
+# Exit with error, even if all the test case has been successful.
+exit 7
+
+% @kbd{cp @var{PREFIX}/share/automake-@var{APIVERSION}/tap-driver.pl .}
+% @kbd{autoreconf -vi && ./configure && make check}
+...
+PASS: foo.test 1 - Swallows fly
+XFAIL: foo.test 2 - Caterpillars fly # TODO metamorphosis in progress
+SKIP: foo.test 3 - Pigs fly # SKIP not enough acid
+PASS: foo.test 4 - Flies fly too :-)
+FAIL: bar.test 1 - Bummer, this test has failed.
+PASS: bar.test 2 - This passed though.
+ERROR: bar.test - Bail out! Ennui kicking in, sorry...
+PASS: baz.test 1
+ERROR: baz.test - exited with status 7
+...
+Please report to bug-automake@@gnu.org
+...
+% @kbd{echo exit status: $?}
+exit status: 1
+
+@c Keep the "skewed" indentation below, it produces pretty PDF output.
+% @kbd{env TEST_LOG_DRIVER_FLAGS='--comments --ignore-exit' \
+ TESTS='foo.test baz.test' make -e check}
+...
+PASS: foo.test 1 - Swallows fly
+XFAIL: foo.test 2 - Caterpillars fly # TODO metamorphosis in progress
+SKIP: foo.test 3 - Pigs fly # SKIP not enough acid
+# foo.test: I just love word plays...
+PASS: foo.test 4 - Flies fly too :-)
+PASS: baz.test 1
+...
+% @kbd{echo exit status: $?}
+exit status: 0
+@end example
+
+@node Incompatibilities with other TAP parsers and drivers
+@subsection Incompatibilities with other TAP parsers and drivers
+
+For implementation or historical reasons, the TAP driver and harness as
+implemented by Automake have some minors incompatibilities with the
+mainstream versions, which you should be aware of.
+
+@itemize @bullet
+@item
+A @code{Bail out!} directive doesn't stop the whole testsuite, but only
+the test script it occurs into. This doesn't follows TAP specifications,
+but on the other hand it maximizes compatibility (and code sharing) with
+the ``hard error'' concept of the default @option{parallel-tests} driver.
+@item
+The @code{version} and @code{pragma} directives are not supported.
+@item
+The @option{--diagnostic-string} option of out driver allows to modify
+the string that introduces TAP diagnostic from the default value
+of ``@code{#}''. The standard TAP protocol has currently no way to
+allow this, so if you use it your diagnostic will be lost to more
+compliant tools like @command{prove} and @code{Test::Harness}
+@item
+And there are probably some other small and yet undiscovered
+incompatibilities, especially in corner cases or with rare usages.
+@end itemize
+
+@node Links and external resources on TAP
+@subsection Links and external resources on TAP
+
+@noindent
+Here are some links to more extensive official or third-party
+documentation and resources about the TAP protocol and related
+tools and libraries.
+@itemize @bullet
+@item
+@uref{http://search.cpan.org/~petdance/Test-Harness/lib/Test/Harness/TAP.pod,
+ @samp{Test::Harness::TAP}},
+the (mostly) official documentation about the TAP format and protocol.
+@item
+@uref{http://search.cpan.org/~andya/Test-Harness/bin/prove,
+ @command{prove}},
+the most famous command-line TAP test driver, included in the distribution
+of @command{perl} and
+@uref{http://search.cpan.org/~andya/Test-Harness/lib/Test/Harness.pm,
+ @samp{Test::Harness}}.
+@item
+The @uref{http://testanything.org/wiki/index.php/Main_Page,TAP wiki}.
+@item
+A ``gentle introduction'' to testing for perl coders:
+@uref{http://search.cpan.org/dist/Test-Simple/lib/Test/Tutorial.pod,
+ @samp{Test::Tutorial}}.
+@item
+@uref{http://search.cpan.org/~mschwern/Test-Simple/lib/Test/Simple.pm,
+ @samp{Test::Simple}}
+and
+@uref{http://search.cpan.org/~mschwern/Test-Simple/lib/Test/More.pm,
+ @samp{Test::More}},
+the standard perl testing libraries, which are based on TAP.
+@item
+@uref{http://www.eyrie.org/~eagle/software/c-tap-harness/,C TAP Harness},
+a C-based project implementing both a TAP producer and a TAP consumer.
+@item
+@uref{http://www.tap4j.org/,tap4j},
+a Java-based project implementing both a TAP producer and a TAP consumer.
+@end itemize
@node DejaGnu Tests
@section DejaGnu Tests
For more information regarding DejaGnu test suites, see @ref{Top, , ,
dejagnu, The DejaGnu Manual}.
-In either case, the testing is done via @samp{make check}.
-
@node Install Tests
@section Install Tests
@item @option{color-tests}
@cindex Option, @option{color-tests}
@opindex color-tests
-Cause output of the simple test suite (@pxref{Simple Tests}) to be
-colorized on capable terminals.
+Cause output of the serial and parallel test harnesses (see @ref{Simple
+Tests}) and of properly-written custom test drivers (@pxref{Custom Test
+Drivers}) to be colorized on capable terminals.
@item @option{dejagnu}
@cindex Option, @option{dejagnu}
@item @option{parallel-tests}
@cindex Option, @option{parallel-tests}
@opindex parallel-tests
-Enable test suite driver for @code{TESTS} that can run tests in parallel
-(@pxref{Simple Tests using parallel-tests}, for more information).
+Enable test suite harness for @code{TESTS} that can run tests in parallel
+(@pxref{Parallel Test Harness}, for more information).
@item @option{readme-alpha}
@cindex Option, @option{readme-alpha}
PRE_UNINSTALL = :
POST_UNINSTALL = :
build_triplet = @build@
+host_triplet = @host@
subdir = lib/Automake
DIST_COMMON = $(dist_perllib_DATA) $(srcdir)/Makefile.am \
$(srcdir)/Makefile.in
docdir = @docdir@
dvidir = @dvidir@
exec_prefix = @exec_prefix@
+host = @host@
host_alias = @host_alias@
+host_cpu = @host_cpu@
+host_os = @host_os@
+host_vendor = @host_vendor@
htmldir = @htmldir@
includedir = @includedir@
infodir = @infodir@
## Process this file with automake to create Makefile.in
-# Copyright (C) 2002, 2003, 2008, 2009 Free Software Foundation, Inc.
+# Copyright (C) 2002, 2003, 2008, 2009, 2010 Free Software Foundation,
+# Inc.
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
Version3.pl
EXTRA_DIST = $(TESTS)
+
+include $(top_srcdir)/CheckListOfTests.am
@SET_MAKE@
-# Copyright (C) 2002, 2003, 2008, 2009 Free Software Foundation, Inc.
+# Copyright (C) 2002, 2003, 2008, 2009, 2010 Free Software Foundation,
+# Inc.
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
PRE_UNINSTALL = :
POST_UNINSTALL = :
build_triplet = @build@
+host_triplet = @host@
+DIST_COMMON = $(srcdir)/Makefile.am $(srcdir)/Makefile.in \
+ $(top_srcdir)/CheckListOfTests.am
subdir = lib/Automake/tests
-DIST_COMMON = $(srcdir)/Makefile.am $(srcdir)/Makefile.in
ACLOCAL_M4 = $(top_srcdir)/aclocal.m4
am__aclocal_m4_deps = $(top_srcdir)/m4/amversion.m4 \
$(top_srcdir)/m4/auxdir.m4 $(top_srcdir)/m4/init.m4 \
am__v_at_0 = @
SOURCES =
DIST_SOURCES =
+am__tty_colors_dummy = \
+ mgn= red= grn= lgn= blu= brg= std=; \
+ am__color_tests=no
# If stdout is a non-dumb tty, use colors. If test -t is not supported,
# then this fails; a conservative approach. Of course do not redirect
# stdout here, just stderr.
am__tty_colors = \
-red=; grn=; lgn=; blu=; std=; \
+$(am__tty_colors_dummy); \
test "X$(AM_COLOR_TESTS)" != Xno \
&& test "X$$TERM" != Xdumb \
&& { test "X$(AM_COLOR_TESTS)" = Xalways || test -t 1 2>/dev/null; } \
&& { \
+ am__color_tests=yes; \
red='\e[0;31m'; \
grn='\e[0;32m'; \
lgn='\e[1;32m'; \
blu='\e[1;34m'; \
+ mgn='\e[0;35m'; \
+ brg='\e[1m'; \
std='\e[m'; \
}
am__vpath_adj_setup = srcdirstrip=`echo "$(srcdir)" | sed 's|.|.|g'`;
# Restructured Text title and section.
am__rst_title = sed 's/.*/ & /;h;s/./=/g;p;x;p;g;p;s/.*//'
am__rst_section = sed 'p;s/./=/g;p;g'
-# Put stdin (possibly several lines separated by ". ") in a box.
-# Prefix each line by 'col' and terminate each with 'std', for coloring.
-# Multi line coloring is problematic with "less -R", so we really need
-# to color each line individually.
-am__text_box = $(AWK) '{ \
- n = split($$0, lines, "\\. "); max = 0; \
- for (i = 1; i <= n; ++i) \
- if (max < length(lines[i])) \
- max = length(lines[i]); \
- for (i = 0; i < max; ++i) \
- line = line "="; \
- print col line std; \
- for (i = 1; i <= n; ++i) \
- if (lines[i]) \
- print col lines[i] std; \
- print col line std; \
-}'
# Solaris 10 'make', and several other traditional 'make' implementations,
# pass "-e" to $(SHELL), and POSIX 2008 even requires this. Work around it
# by disabling -e (using the XSI extension "set +e") if it's set.
am__sh_e_setup = case $$- in *e*) set +e;; esac
+# Default flags passed to test drivers.
+am__common_driver_flags = \
+ --color-tests "$$am__color_tests" \
+ --enable-hard-errors "$$am__enable_hard_errors" \
+ --expect-failure "$$am__expect_failure"
# To be inserted before the command running the test. Creates the
# directory for the log if needed. Stores in $dir the directory
# containing $f, in $tst the test, in $log the log. Executes the
# developer- defined test setup AM_TESTS_ENVIRONMENT (if any), and
-# passes TESTS_ENVIRONMENT. Saves and restores TERM around uses of
-# TESTS_ENVIRONMENT and AM_TESTS_ENVIRONMENT, in case any of them
-# unsets it.
+# passes TESTS_ENVIRONMENT. Set up options for the wrapper that
+# will run the test scripts (or their associated LOG_COMPILER, if
+# thy have one).
am__check_pre = \
$(am__sh_e_setup); \
$(am__vpath_adj_setup) $(am__vpath_adj) \
+$(am__tty_colors); \
srcdir=$(srcdir); export srcdir; \
-rm -f $@-t; \
-am__trap='rm -f '\''$(abs_builddir)/$@-t'\''; (exit $$st); exit $$st'; \
-trap "st=129; $$am__trap" 1; trap "st=130; $$am__trap" 2; \
-trap "st=141; $$am__trap" 13; trap "st=143; $$am__trap" 15; \
am__odir=`echo "./$@" | sed 's|/[^/]*$$||'`; \
test "x$$am__odir" = x. || $(MKDIR_P) "$$am__odir" || exit $$?; \
if test -f "./$$f"; then dir=./; \
elif test -f "$$f"; then dir=; \
else dir="$(srcdir)/"; fi; \
-tst=$$dir$$f; log='$@'; __SAVED_TERM=$$TERM; \
-$(AM_TESTS_ENVIRONMENT) \
-$(TESTS_ENVIRONMENT)
-# To be appended to the command running the test. Handle the stdout
-# and stderr redirection, and catch the exit status.
-am__check_post = \
->$@-t 2>&1; \
-estatus=$$?; \
-if test -n '$(DISABLE_HARD_ERRORS)' \
- && test $$estatus -eq 99; then \
- estatus=1; \
-fi; \
-TERM=$$__SAVED_TERM; export TERM; \
-$(am__tty_colors); \
-xfailed=PASS; \
+tst=$$dir$$f; log='$@'; \
+if test -n '$(DISABLE_HARD_ERRORS)'; then \
+ am__enable_hard_errors=no; \
+else \
+ am__enable_hard_errors=yes; \
+fi; \
case " $(XFAIL_TESTS) " in \
*[\ \ ]$$f[\ \ ]* | *[\ \ ]$$dir$$f[\ \ ]*) \
- xfailed=XFAIL;; \
-esac; \
-case $$estatus.$$xfailed in \
- 0.XFAIL) col=$$red; res=XPASS;; \
- 0.*) col=$$grn; res=PASS ;; \
- 77.*) col=$$blu; res=SKIP ;; \
- 99.*) col=$$red; res=FAIL ;; \
- *.XFAIL) col=$$lgn; res=XFAIL;; \
- *.*) col=$$red; res=FAIL ;; \
-esac; \
-echo "$${col}$$res$${std}: $$f"; \
-echo "$$res: $$f (exit: $$estatus)" | \
- $(am__rst_section) >$@; \
-cat $@-t >>$@; \
-rm -f $@-t
+ am__expect_failure=yes;; \
+ *) \
+ am__expect_failure=no;; \
+esac; \
+$(AM_TESTS_ENVIRONMENT) $(TESTS_ENVIRONMENT)
+# A shell command to get the names of the tests scripts with any registered
+# extension removed (i.e., equivalently, the names of the test logs, with
+# the `.log' extension removed). The result is saved in the shell variable
+# `$bases'. This honors runtime overriding of TESTS and TEST_LOGS. Sadly,
+# we cannot use something simpler, involving e.g., "$(TEST_LOGS:.log=)",
+# since that might cause problem with VPATH rewrites for suffix-less tests.
+# See also 'test-harness-vpath-rewrite.test' and 'test-trs-basic.test'.
+am__set_TESTS_bases = \
+ bases='$(TEST_LOGS)'; \
+ bases=`for i in $$bases; do echo $$i; done | sed 's/\.log$$//'`; \
+ bases=`echo $$bases`
RECHECK_LOGS = $(TEST_LOGS)
AM_RECURSIVE_TARGETS = check check-html recheck recheck-html
TEST_SUITE_LOG = test-suite.log
TEST_SUITE_HTML = $(TEST_SUITE_LOG:.log=.html)
am__test_logs1 = $(TESTS:=.log)
TEST_LOGS = $(am__test_logs1:.pl.log=.log)
+PL_LOG_DRIVER = $(SHELL) $(top_srcdir)/lib/test-driver
PL_LOG_COMPILE = $(PL_LOG_COMPILER) $(AM_PL_LOG_FLAGS) $(PL_LOG_FLAGS)
-TEST_LOGS_TMP = $(TEST_LOGS:.log=.log-t)
DISTFILES = $(DIST_COMMON) $(DIST_SOURCES) $(TEXINFOS) $(EXTRA_DIST)
ACLOCAL = @ACLOCAL@
AMTAR = @AMTAR@
docdir = @docdir@
dvidir = @dvidir@
exec_prefix = @exec_prefix@
+host = @host@
host_alias = @host_alias@
+host_cpu = @host_cpu@
+host_os = @host_os@
+host_vendor = @host_vendor@
htmldir = @htmldir@
includedir = @includedir@
infodir = @infodir@
Version3.pl
EXTRA_DIST = $(TESTS)
+am__tmk = tests-in-makefile-list.tmp
+am__tfs = tests-on-filesystem-list.tmp
+am__tdf = diff-in-tests-lists.tmp
all: all-am
.SUFFIXES:
-.SUFFIXES: .html .log .pl
-$(srcdir)/Makefile.in: $(srcdir)/Makefile.am $(am__configure_deps)
+.SUFFIXES: .html .log .pl .trs
+$(srcdir)/Makefile.in: $(srcdir)/Makefile.am $(top_srcdir)/CheckListOfTests.am $(am__configure_deps)
@for dep in $?; do \
case '$(am__configure_deps)' in \
*$$dep*) \
cscope cscopelist:
+# Recover from deleted `.trs' file; this should ensure that
+# "rm -f foo.log; make foo.trs" re-run `foo.test', and re-create
+# both `foo.log' and `foo.trs'. Break the recipe in two subshells
+# to avoid problems with "make -n".
+.log.trs:
+ rm -f $< $@
+ $(MAKE) $(AM_MAKEFLAGS) $<
+
+# Helper recipe used by $(TEST_SUITE_LOG) below, to avoid problems with
+# "make -n". Break this recipe in multiple shell invocations too, to
+# really work as expected with "make -n".
+am--redo-logs:
+ @rm -f $$redo_logs
+ @rm -f $$redo_results
+ @if test -n "$$am__remaking_logs"; then \
+ echo "fatal: making $(TEST_SUITE_LOG): possible infinite" \
+ "recursion detected" >&2; \
+ else \
+ am__remaking_logs=yes $(MAKE) $(AM_MAKEFLAGS) $$redo_logs; \
+ fi;
+ @st=0; \
+ errmsg="fatal: making $(TEST_SUITE_LOG): failed to create"; \
+ for i in $$redo_bases; do \
+ test -f $$i.trs && test -r $$i.trs \
+ || { echo "$$errmsg $$i.trs"; st=1; }; \
+ test -f $$i.log && test -r $$i.log \
+ || { echo "$$errmsg $$i.log"; st=1; }; \
+ done; \
+ test $$st -eq 0
+
$(TEST_SUITE_LOG): $(TEST_LOGS)
- @$(am__sh_e_setup); \
- list='$(TEST_LOGS)'; \
- results=`for f in $$list; do \
- test -r $$f && read line < $$f && echo "$$line" \
- || echo FAIL; \
- done`; \
- all=`echo "$$results" | sed '/^$$/d' | wc -l | sed -e 's/^[ ]*//'`; \
- fail=`echo "$$results" | grep -c '^FAIL'`; \
- pass=`echo "$$results" | grep -c '^PASS'`; \
- skip=`echo "$$results" | grep -c '^SKIP'`; \
- xfail=`echo "$$results" | grep -c '^XFAIL'`; \
- xpass=`echo "$$results" | grep -c '^XPASS'`; \
- failures=`expr $$fail + $$xpass`; \
- all=`expr $$all - $$skip`; \
- if test "$$all" -eq 1; then tests=test; All=; \
- else tests=tests; All="All "; fi; \
- case fail=$$fail:xpass=$$xpass:xfail=$$xfail in \
- fail=0:xpass=0:xfail=0) \
- msg="$$All$$all $$tests passed. "; \
- exit=true;; \
- fail=0:xpass=0:xfail=*) \
- msg="$$All$$all $$tests behaved as expected"; \
- if test "$$xfail" -eq 1; then xfailures=failure; \
- else xfailures=failures; fi; \
- msg="$$msg ($$xfail expected $$xfailures). "; \
- exit=true;; \
- fail=*:xpass=0:xfail=*) \
- msg="$$fail of $$all $$tests failed. "; \
- exit=false;; \
- fail=*:xpass=*:xfail=*) \
- msg="$$failures of $$all $$tests did not behave as expected"; \
- if test "$$xpass" -eq 1; then xpasses=pass; \
- else xpasses=passes; fi; \
- msg="$$msg ($$xpass unexpected $$xpasses). "; \
- exit=false;; \
- *) \
- echo >&2 "incorrect case"; exit 4;; \
- esac; \
- if test "$$skip" -ne 0; then \
- if test "$$skip" -eq 1; then \
- msg="$$msg($$skip test was not run). "; \
- else \
- msg="$$msg($$skip tests were not run). "; \
- fi; \
- fi; \
+ @$(am__set_TESTS_bases); \
+ am__f_ok () { test -f "$$1" && test -r "$$1"; }; \
+ redo_bases=`for i in $$bases; do \
+ am__f_ok $$i.trs && am__f_ok $$i.log || echo $$i; \
+ done`; \
+ if test -n "$$redo_bases"; then \
+ redo_logs=`for i in $$redo_bases; do echo $$i.log; done`; \
+ redo_results=`for i in $$redo_bases; do echo $$i.trs; done`; \
+ redo_bases="$$redo_bases" \
+ redo_logs="$$redo_logs" \
+ redo_results="$$redo_results" \
+ $(MAKE) $(AM_MAKEFLAGS) am--redo-logs || exit 1; \
+ else :; fi;
+ @$(am__sh_e_setup); $(am__tty_colors); $(am__set_TESTS_bases); \
+ ws='[ ]'; \
+ results=`for b in $$bases; do echo $$b.trs; done`; \
+ test -n "$$results" || results=/dev/null; \
+ all=` grep "^$$ws*:test-result:" $$results | wc -l`; \
+ pass=` grep "^$$ws*:test-result:$$ws*PASS" $$results | wc -l`; \
+ fail=` grep "^$$ws*:test-result:$$ws*FAIL" $$results | wc -l`; \
+ skip=` grep "^$$ws*:test-result:$$ws*SKIP" $$results | wc -l`; \
+ xfail=`grep "^$$ws*:test-result:$$ws*XFAIL" $$results | wc -l`; \
+ xpass=`grep "^$$ws*:test-result:$$ws*XPASS" $$results | wc -l`; \
+ error=`grep "^$$ws*:test-result:$$ws*ERROR" $$results | wc -l`; \
+ if test `expr $$fail + $$xpass + $$error` -eq 0; then \
+ success=true; \
+ else \
+ success=false; \
+ fi; \
+ br='==================='; br=$$br$$br$$br$$br; \
+ result_count () \
+ { \
+ if test x"$$1" = x"--maybe-color"; then \
+ maybe_colorize=yes; \
+ elif test x"$$1" = x"--no-color"; then \
+ maybe_colorize=no; \
+ else \
+ echo "$@: invalid 'result_count' usage" >&2; exit 4; \
+ fi; \
+ shift; \
+ desc=$$1 count=$$2; \
+ if test $$maybe_colorize = yes && test $$count -gt 0; then \
+ color_start=$$3 color_end=$$std; \
+ else \
+ color_start= color_end=; \
+ fi; \
+ echo "$${color_start}# $$desc $$count$${color_end}"; \
+ }; \
+ create_testsuite_report () \
+ { \
+ result_count $$1 "TOTAL:" $$all "$$brg"; \
+ result_count $$1 "PASS: " $$pass "$$grn"; \
+ result_count $$1 "SKIP: " $$skip "$$blu"; \
+ result_count $$1 "XFAIL:" $$xfail "$$lgn"; \
+ result_count $$1 "FAIL: " $$fail "$$red"; \
+ result_count $$1 "XPASS:" $$xpass "$$red"; \
+ result_count $$1 "ERROR:" $$error "$$mgn"; \
+ }; \
{ \
echo "$(PACKAGE_STRING): $(subdir)/$(TEST_SUITE_LOG)" | \
$(am__rst_title); \
- echo "$$msg"; \
+ create_testsuite_report --no-color; \
echo; \
echo ".. contents:: :depth: 2"; \
echo; \
- for f in $$list; do \
- test -r $$f && read line < $$f || line=; \
- case $$line in \
- PASS:*|XFAIL:*);; \
- *) echo; cat $$f;; \
- esac; \
- done; \
+ for i in $$bases; do \
+ if grep "^$$ws*:copy-in-global-log:$$ws*no$$ws*$$" $$i.trs \
+ >/dev/null; then continue; \
+ fi; \
+ glob_res=`sed -n -e "s/$$ws*$$//" \
+ -e "s/^$$ws*:global-test-result:$$ws*//p" \
+ $$i.trs`; \
+ test -n "$$glob_res" || glob_res=RUN; \
+ echo "$$glob_res: $$i" | $(am__rst_section); \
+ if test ! -r $$i.log; then \
+ echo "fatal: making $@: $$i.log is unreadable" >&2; \
+ exit 1; \
+ fi; \
+ cat $$i.log; echo; \
+ done; \
} >$(TEST_SUITE_LOG).tmp; \
mv $(TEST_SUITE_LOG).tmp $(TEST_SUITE_LOG); \
- if test "$$failures" -ne 0; then \
- msg="$${msg}See $(subdir)/$(TEST_SUITE_LOG). "; \
- if test -n "$(PACKAGE_BUGREPORT)"; then \
- msg="$${msg}Please report to $(PACKAGE_BUGREPORT). "; \
- fi; \
- fi; \
- test x"$$VERBOSE" = x || $$exit || cat $(TEST_SUITE_LOG); \
- $(am__tty_colors); \
- if $$exit; then \
+ if $$success; then \
col="$$grn"; \
else \
col="$$red"; \
+ test x"$$VERBOSE" = x || cat $(TEST_SUITE_LOG); \
fi; \
- echo "$$msg" | $(am__text_box) "col=$$col" "std=$$std"; \
- $$exit || exit 1
+ echo "$${col}$$br$${std}"; \
+ echo "$${col}Testsuite summary for $(PACKAGE_STRING)$${std}"; \
+ echo "$${col}$$br$${std}"; \
+ create_testsuite_report --maybe-color; \
+ echo "$$col$$br$$std"; \
+ if $$success; then :; else \
+ echo "$${col}See $(subdir)/$(TEST_SUITE_LOG)$${std}"; \
+ if test -n "$(PACKAGE_BUGREPORT)"; then \
+ echo "$${col}Please report to $(PACKAGE_BUGREPORT)$${std}"; \
+ fi; \
+ echo "$$col$$br$$std"; \
+ fi; \
+ $$success || exit 1
# Run all the tests.
check-TESTS:
@list='$(RECHECK_LOGS)'; test -z "$$list" || rm -f $$list
+ @list='$(RECHECK_LOGS:.log=.trs)'; test -z "$$list" || rm -f $$list
@test -z "$(TEST_SUITE_LOG)" || rm -f $(TEST_SUITE_LOG)
- @list='$(TEST_LOGS)'; \
- list=`for f in $$list; do \
- test .log = $$f || echo $$f; \
- done | tr '\012\015' ' '`; \
+ @list='$(TEST_LOGS)'; \
+ list=`for i in $$list; do \
+ test .log = $$i || echo $$i; \
+ done | tr '\012\015' ' '`; \
+ list=`echo "$$list" | sed 's/ *$$//'`; \
$(MAKE) $(AM_MAKEFLAGS) $(TEST_SUITE_LOG) TEST_LOGS="$$list"
.log.html:
$(MAKE) $(AM_MAKEFLAGS) $(TEST_SUITE_HTML) || exit 4; \
exit $$rv
recheck recheck-html:
- @target=`echo $@ | sed 's,^re,,'`; \
- list='$(TEST_LOGS)'; \
- list=`for f in $$list; do \
- test -f $$f || continue; \
- if test -r $$f && read line < $$f; then \
- case $$line in FAIL*|XPASS*) echo $$f;; esac; \
- else echo $$f; fi; \
+ @ws='[ ]'; \
+ target=`echo $@ | sed 's,^re,,'`; \
+ $(am__set_TESTS_bases); \
+ list=`for i in $$bases; do \
+ test -f $$i.trs || test -f $$i.log || continue; \
+ grep "^$$ws*:recheck:$$ws*no$$ws*$$" $$i.trs \
+ >/dev/null 2>&1 || echo $$i.log; \
done | tr '\012\015' ' '`; \
list=`echo "$$list" | sed 's/ *$$//'`; \
$(MAKE) $(AM_MAKEFLAGS) $$target AM_MAKEFLAGS='$(AM_MAKEFLAGS) TEST_LOGS="'"$$list"'"'
.pl.log:
- @p='$<'; $(am__check_pre) $(PL_LOG_COMPILE) "$$tst" $(am__check_post)
+ @p='$<'; $(am__check_pre) $(PL_LOG_DRIVER) --test-name "$$f" \
+ --log-file '$*.log' --trs-file '$*.trs' \
+ $(am__common_driver_flags) $(AM_PL_LOG_DRIVER_FLAGS) $(PL_LOG_DRIVER_FLAGS) -- $(PL_LOG_COMPILE) "$$tst" \
+ $(AM_TESTS_FD_REDIRECT)
distdir: $(DISTFILES)
@srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \
fi
mostlyclean-generic:
-test -z "$(TEST_LOGS)" || rm -f $(TEST_LOGS)
- -test -z "$(TEST_LOGS_TMP)" || rm -f $(TEST_LOGS_TMP)
+ -test -z "$(TEST_LOGS:.log=.trs)" || rm -f $(TEST_LOGS:.log=.trs)
-test -z "$(TEST_SUITE_HTML)" || rm -f $(TEST_SUITE_HTML)
-test -z "$(TEST_SUITE_LOG)" || rm -f $(TEST_SUITE_LOG)
@echo "it deletes files that may require special tools to rebuild."
clean: clean-am
-clean-am: clean-generic mostlyclean-am
+clean-am: clean-generic clean-local mostlyclean-am
distclean: distclean-am
-rm -f Makefile
.MAKE: check-am check-html install-am install-strip recheck \
recheck-html
-.PHONY: all all-am check check-TESTS check-am check-html clean \
- clean-generic distclean distclean-generic distdir dvi dvi-am \
- html html-am info info-am install install-am install-data \
- install-data-am install-dvi install-dvi-am install-exec \
- install-exec-am install-html install-html-am install-info \
- install-info-am install-man install-pdf install-pdf-am \
- install-ps install-ps-am install-strip installcheck \
- installcheck-am installdirs maintainer-clean \
- maintainer-clean-generic mostlyclean mostlyclean-generic pdf \
- pdf-am ps ps-am recheck recheck-html uninstall uninstall-am
-
+.PHONY: all all-am am--redo-logs check check-TESTS check-am check-html \
+ clean clean-generic clean-local distclean distclean-generic \
+ distdir dvi dvi-am html html-am info info-am install \
+ install-am install-data install-data-am install-dvi \
+ install-dvi-am install-exec install-exec-am install-html \
+ install-html-am install-info install-info-am install-man \
+ install-pdf install-pdf-am install-ps install-ps-am \
+ install-strip installcheck installcheck-am installdirs \
+ maintainer-clean maintainer-clean-generic mostlyclean \
+ mostlyclean-generic pdf pdf-am ps ps-am recheck recheck-html \
+ uninstall uninstall-am
+
+
+.PHONY: maintainer-check-list-of-tests
+maintainer-check-list-of-tests:
+ @if diff -u /dev/null /dev/null >/dev/null 2>&1; then \
+ diff='diff -u'; \
+ else \
+ diff='diff'; \
+ fi; \
+ set X $(checked_test_extensions); shift; \
+ if test $$# -eq 0; then \
+ set X $(TEST_EXTENSIONS); shift; \
+ fi; \
+ if test $$# -eq 0; then \
+ echo '$@: internal error: $$(checked_test_extensions) and' \
+ '$$(TEST_EXTENSIONS) are both undefined or empty' >&2; \
+ exit 255; \
+ fi; \
+ lst='$(expected_list_of_tests)'; \
+ test -n "$$lst" || lst='$(TESTS)'; \
+ for t in $$lst; do \
+ echo "$$t"; \
+ done | sort >$(am__tmk); \
+ for ext in $$*; do \
+ ls *$$ext 2>/dev/null; \
+ if test $(srcdir) != $(builddir); then \
+ (cd $(srcdir) && ls *$$ext 2>/dev/null); \
+ fi; \
+ done | sort | uniq >$(am__tfs); \
+ if $$diff $(am__tmk) $(am__tfs) >$(am__tdf); then \
+ result=0; \
+ else \
+ echo '$@: list of tests in Makefile an on filesystem differ' >&2; \
+ echo "+ $$diff in-makefile on-filesystem" >&2; \
+ cat $(am__tdf) >&2; \
+ result=1; \
+ fi; \
+ rm -f $(am__tmk) $(am__tfs) $(am__tdf); \
+ exit $$result;
+
+.PHONY: clean-maintcheck-testslist-tmp
+clean-local: clean-maintcheck-testslist-tmp
+clean-maintcheck-testslist-tmp:
+ rm -f $(am__tmk) $(am__tfs) $(am__tdf)
# Tell versions [3.59,3.63) of GNU make to not export all variables.
# Otherwise a system limit (for SysV at least) may be exceeded.
## _SCRIPTS, then the program transform will be applied, which is not
## what we want. So we make them executable by hand.
scriptdir = $(pkgvdatadir)
-dist_script_DATA = config.guess config.sub install-sh mdate-sh missing \
- mkinstalldirs elisp-comp ylwrap acinstall depcomp compile py-compile \
- symlink-tree ar-lib
+dist_script_DATA = \
+ config.guess \
+ config.sub \
+ install-sh \
+ mdate-sh \
+ missing \
+ mkinstalldirs \
+ elisp-comp \
+ ylwrap \
+ acinstall \
+ depcomp \
+ compile \
+ py-compile \
+ symlink-tree \
+ ar-lib \
+ test-driver \
+ tap-driver.sh \
+ tap-driver.pl
EXTRA_DIST = gnupload
PRE_UNINSTALL = :
POST_UNINSTALL = :
build_triplet = @build@
+host_triplet = @host@
subdir = lib
DIST_COMMON = $(dist_pkgvdata_DATA) $(dist_script_DATA) \
$(srcdir)/Makefile.am $(srcdir)/Makefile.in COPYING INSTALL \
docdir = @docdir@
dvidir = @dvidir@
exec_prefix = @exec_prefix@
+host = @host@
host_alias = @host_alias@
+host_cpu = @host_cpu@
+host_os = @host_os@
+host_vendor = @host_vendor@
htmldir = @htmldir@
includedir = @includedir@
infodir = @infodir@
SUBDIRS = Automake am
dist_pkgvdata_DATA = COPYING INSTALL texinfo.tex config-ml.in
scriptdir = $(pkgvdatadir)
-dist_script_DATA = config.guess config.sub install-sh mdate-sh missing \
- mkinstalldirs elisp-comp ylwrap acinstall depcomp compile py-compile \
- symlink-tree ar-lib
+dist_script_DATA = \
+ config.guess \
+ config.sub \
+ install-sh \
+ mdate-sh \
+ missing \
+ mkinstalldirs \
+ elisp-comp \
+ ylwrap \
+ acinstall \
+ depcomp \
+ compile \
+ py-compile \
+ symlink-tree \
+ ar-lib \
+ test-driver \
+ tap-driver.sh \
+ tap-driver.pl
EXTRA_DIST = gnupload
all: all-recursive
PRE_UNINSTALL = :
POST_UNINSTALL = :
build_triplet = @build@
+host_triplet = @host@
subdir = lib/am
DIST_COMMON = $(dist_am_DATA) $(srcdir)/Makefile.am \
$(srcdir)/Makefile.in
docdir = @docdir@
dvidir = @dvidir@
exec_prefix = @exec_prefix@
+host = @host@
host_alias = @host_alias@
+host_cpu = @host_cpu@
+host_os = @host_os@
+host_vendor = @host_vendor@
htmldir = @htmldir@
includedir = @includedir@
infodir = @infodir@
## You should have received a copy of the GNU General Public License
## along with this program. If not, see <http://www.gnu.org/licenses/>.
+am__tty_colors_dummy = \
+ mgn= red= grn= lgn= blu= brg= std=; \
+ am__color_tests=no
if %?COLOR%
# If stdout is a non-dumb tty, use colors. If test -t is not supported,
# then this fails; a conservative approach. Of course do not redirect
# stdout here, just stderr.
am__tty_colors = \
-red=; grn=; lgn=; blu=; std=; \
+$(am__tty_colors_dummy); \
test "X$(AM_COLOR_TESTS)" != Xno \
&& test "X$$TERM" != Xdumb \
&& { test "X$(AM_COLOR_TESTS)" = Xalways || test -t 1 2>/dev/null; } \
&& { \
+ am__color_tests=yes; \
red='\e[0;31m'; \
grn='\e[0;32m'; \
lgn='\e[1;32m'; \
blu='\e[1;34m'; \
+ mgn='\e[0;35m'; \
+ brg='\e[1m'; \
std='\e[m'; \
}
else !%?COLOR%
-am__tty_colors = \
-red=; grn=; lgn=; blu=; std=
+am__tty_colors = $(am__tty_colors_dummy)
endif !%?COLOR%
.PHONY: check-TESTS
## New parallel test driver.
##
-## This code is adapted from check.mk which was originally
-## written at EPITA/LRDE, further developed at Gostai, then made
-## its way from GNU coreutils to end up, largely rewritten, in
-## Automake.
-##
-## It provides special support for "unit tests", that is to say,
-## tests that (once run) no longer need to be re-compiled and
-## re-run at each "make check", unless their sources changed. To
-## enable unit-test supports, set RECHECK_LOGS to empty. In such a
-## setting, that heavily relies on correct dependencies, its users may
-## prefer to define EXTRA_PROGRAMS instead of check_PROGRAMS, because
-## it allows intertwined compilation and execution of the tests.
-## Sometimes this helps catching errors earlier (you don't have to
-## wait for all the tests to be compiled).
-##
-## Define TEST_SUITE_LOG to be the name of the global log to create.
-## Define TEST_LOGS to the set of logs to include in it. It defaults
-## to $(TESTS), with `.test' and `@EXEEXT@' removed, and `'.log'
-## appended.
-##
-## In addition to the magic "exit 77 means SKIP" feature (which was
-## imported from automake), there is a magic "exit 99 means FAIL" feature
-## which is useful if you need to issue a hard error no matter whether the
-## test is XFAIL or not. You can disable this feature by setting the
-## variable DISABLE_HARD_ERRORS to a nonempty value.
+## The first version of the code here was adapted from check.mk, which was
+## originally written at EPITA/LRDE, further developed at Gostai, then made
+## its way from GNU coreutils to end up, largely rewritten, in Automake.
+## The current version is an heavy rewrite of that, to allow for support
+## of more test metadata, and the use of custom test derivers and protocols
+## (among them, TAP).
# Restructured Text title and section.
-am__rst_title = sed 's/.*/ & /;h;s/./=/g;p;x;p;g;p;s/.*//'
+am__rst_title = sed 's/.*/ & /;h;s/./=/g;p;x;p;g;p;s/.*//'
am__rst_section = sed 'p;s/./=/g;p;g'
-# Put stdin (possibly several lines separated by ". ") in a box.
-# Prefix each line by 'col' and terminate each with 'std', for coloring.
-# Multi line coloring is problematic with "less -R", so we really need
-# to color each line individually.
-am__text_box = $(AWK) '{ \
- n = split($$0, lines, "\\. "); max = 0; \
- for (i = 1; i <= n; ++i) \
- if (max < length(lines[i])) \
- max = length(lines[i]); \
- for (i = 0; i < max; ++i) \
- line = line "="; \
- print col line std; \
- for (i = 1; i <= n; ++i) \
- if (lines[i]) \
- print col lines[i] std; \
- print col line std; \
-}'
-
# Solaris 10 'make', and several other traditional 'make' implementations,
# pass "-e" to $(SHELL), and POSIX 2008 even requires this. Work around it
# by disabling -e (using the XSI extension "set +e") if it's set.
am__sh_e_setup = case $$- in *e*) set +e;; esac
+# Default flags passed to test drivers.
+am__common_driver_flags = \
+ --color-tests "$$am__color_tests" \
+ --enable-hard-errors "$$am__enable_hard_errors" \
+ --expect-failure "$$am__expect_failure"
+
# To be inserted before the command running the test. Creates the
# directory for the log if needed. Stores in $dir the directory
# containing $f, in $tst the test, in $log the log. Executes the
# developer- defined test setup AM_TESTS_ENVIRONMENT (if any), and
-# passes TESTS_ENVIRONMENT. Saves and restores TERM around uses of
-# TESTS_ENVIRONMENT and AM_TESTS_ENVIRONMENT, in case any of them
-# unsets it.
+# passes TESTS_ENVIRONMENT. Set up options for the wrapper that
+# will run the test scripts (or their associated LOG_COMPILER, if
+# thy have one).
am__check_pre = \
$(am__sh_e_setup); \
$(am__vpath_adj_setup) $(am__vpath_adj) \
+$(am__tty_colors); \
srcdir=$(srcdir); export srcdir; \
-rm -f $@-t; \
-am__trap='rm -f '\''$(abs_builddir)/$@-t'\''; (exit $$st); exit $$st'; \
-trap "st=129; $$am__trap" 1; trap "st=130; $$am__trap" 2; \
-trap "st=141; $$am__trap" 13; trap "st=143; $$am__trap" 15; \
am__odir=`echo "./$@" | sed 's|/[^/]*$$||'`; \
test "x$$am__odir" = x. || $(MKDIR_P) "$$am__odir" || exit $$?; \
if test -f "./$$f"; then dir=./; \
elif test -f "$$f"; then dir=; \
else dir="$(srcdir)/"; fi; \
-tst=$$dir$$f; log='$@'; __SAVED_TERM=$$TERM; \
-$(AM_TESTS_ENVIRONMENT) \
-$(TESTS_ENVIRONMENT)
-
-# To be appended to the command running the test. Handle the stdout
-# and stderr redirection, and catch the exit status.
-am__check_post = \
->$@-t 2>&1; \
-estatus=$$?; \
-if test -n '$(DISABLE_HARD_ERRORS)' \
- && test $$estatus -eq 99; then \
- estatus=1; \
-fi; \
-TERM=$$__SAVED_TERM; export TERM; \
-$(am__tty_colors); \
-xfailed=PASS; \
+tst=$$dir$$f; log='$@'; \
+if test -n '$(DISABLE_HARD_ERRORS)'; then \
+ am__enable_hard_errors=no; \
+else \
+ am__enable_hard_errors=yes; \
+fi; \
+## The use of $dir below is required to account for VPATH
+## rewriting done by Sun make.
case " $(XFAIL_TESTS) " in \
*[\ \ ]$$f[\ \ ]* | *[\ \ ]$$dir$$f[\ \ ]*) \
- xfailed=XFAIL;; \
-esac; \
-case $$estatus.$$xfailed in \
- 0.XFAIL) col=$$red; res=XPASS;; \
- 0.*) col=$$grn; res=PASS ;; \
- 77.*) col=$$blu; res=SKIP ;; \
- 99.*) col=$$red; res=FAIL ;; \
- *.XFAIL) col=$$lgn; res=XFAIL;; \
- *.*) col=$$red; res=FAIL ;; \
-esac; \
-echo "$${col}$$res$${std}: $$f"; \
-echo "$$res: $$f (exit: $$estatus)" | \
- $(am__rst_section) >$@; \
-cat $@-t >>$@; \
-rm -f $@-t
+ am__expect_failure=yes;; \
+ *) \
+ am__expect_failure=no;; \
+esac; \
+$(AM_TESTS_ENVIRONMENT) $(TESTS_ENVIRONMENT)
+
+# A shell command to get the names of the tests scripts with any registered
+# extension removed (i.e., equivalently, the names of the test logs, with
+# the `.log' extension removed). The result is saved in the shell variable
+# `$bases'. This honors runtime overriding of TESTS and TEST_LOGS. Sadly,
+# we cannot use something simpler, involving e.g., "$(TEST_LOGS:.log=)",
+# since that might cause problem with VPATH rewrites for suffix-less tests.
+# See also 'test-harness-vpath-rewrite.test' and 'test-trs-basic.test'.
+am__set_TESTS_bases = \
+ bases='$(TEST_LOGS)'; \
+ bases=`for i in $$bases; do echo $$i; done | sed 's/\.log$$//'`; \
+## Trim away any extra whitespace. This has already proved useful in
+## avoiding weird bug on lesser make implementations.
+ bases=`echo $$bases`
+
+# Recover from deleted `.trs' file; this should ensure that
+# "rm -f foo.log; make foo.trs" re-run `foo.test', and re-create
+# both `foo.log' and `foo.trs'. Break the recipe in two subshells
+# to avoid problems with "make -n".
+.log.trs:
+ rm -f $< $@
+ $(MAKE) $(AM_MAKEFLAGS) $<
+
+# Helper recipe used by $(TEST_SUITE_LOG) below, to avoid problems with
+# "make -n". Break this recipe in multiple shell invocations too, to
+# really work as expected with "make -n".
+am--redo-logs:
+ @rm -f $$redo_logs
+ @rm -f $$redo_results
+## The use of the `am__remaking_logs' environment variable below is
+## required to ensure that we don't go into an infinite recursion in
+## case a test log in $(TEST_LOGS) is the same as $(TEST_SUITE_LOG).
+## Yes, this has already happened in practice. Sigh!
+ @if test -n "$$am__remaking_logs"; then \
+ echo "fatal: making $(TEST_SUITE_LOG): possible infinite" \
+ "recursion detected" >&2; \
+ else \
+ am__remaking_logs=yes $(MAKE) $(AM_MAKEFLAGS) $$redo_logs; \
+ fi;
+## Sanity check: each unreadable or non-existent test result file should
+## has been properly remade at this point, as should the corresponding log
+## file.
+ @st=0; \
+ errmsg="fatal: making $(TEST_SUITE_LOG): failed to create"; \
+ for i in $$redo_bases; do \
+ test -f $$i.trs && test -r $$i.trs \
+ || { echo "$$errmsg $$i.trs"; st=1; }; \
+ test -f $$i.log && test -r $$i.log \
+ || { echo "$$errmsg $$i.log"; st=1; }; \
+ done; \
+ test $$st -eq 0
+.PHONY: am--redo-logs
$(TEST_SUITE_LOG): $(TEST_LOGS)
- @$(am__sh_e_setup); \
- list='$(TEST_LOGS)'; \
- results=`for f in $$list; do \
- test -r $$f && read line < $$f && echo "$$line" \
- || echo FAIL; \
- done`; \
- all=`echo "$$results" | sed '/^$$/d' | wc -l | sed -e 's/^[ ]*//'`; \
- fail=`echo "$$results" | grep -c '^FAIL'`; \
- pass=`echo "$$results" | grep -c '^PASS'`; \
- skip=`echo "$$results" | grep -c '^SKIP'`; \
- xfail=`echo "$$results" | grep -c '^XFAIL'`; \
- xpass=`echo "$$results" | grep -c '^XPASS'`; \
- failures=`expr $$fail + $$xpass`; \
- all=`expr $$all - $$skip`; \
- if test "$$all" -eq 1; then tests=test; All=; \
- else tests=tests; All="All "; fi; \
- case fail=$$fail:xpass=$$xpass:xfail=$$xfail in \
- fail=0:xpass=0:xfail=0) \
- msg="$$All$$all $$tests passed. "; \
- exit=true;; \
- fail=0:xpass=0:xfail=*) \
- msg="$$All$$all $$tests behaved as expected"; \
- if test "$$xfail" -eq 1; then xfailures=failure; \
- else xfailures=failures; fi; \
- msg="$$msg ($$xfail expected $$xfailures). "; \
- exit=true;; \
- fail=*:xpass=0:xfail=*) \
- msg="$$fail of $$all $$tests failed. "; \
- exit=false;; \
- fail=*:xpass=*:xfail=*) \
- msg="$$failures of $$all $$tests did not behave as expected"; \
- if test "$$xpass" -eq 1; then xpasses=pass; \
- else xpasses=passes; fi; \
- msg="$$msg ($$xpass unexpected $$xpasses). "; \
- exit=false;; \
- *) \
- echo >&2 "incorrect case"; exit 4;; \
- esac; \
- if test "$$skip" -ne 0; then \
- if test "$$skip" -eq 1; then \
- msg="$$msg($$skip test was not run). "; \
- else \
- msg="$$msg($$skip tests were not run). "; \
- fi; \
- fi; \
+ @$(am__set_TESTS_bases); \
+## Helper shell function, tells whether a path refers to an existing,
+## regular, readable file.
+ am__f_ok () { test -f "$$1" && test -r "$$1"; }; \
+## We need to ensures that all the required `.trs' and `.log' files will
+## be present and readable. The direct dependencies of $(TEST_SUITE_LOG)
+## only ensure that all the `.log' files exists; they don't ensure that
+## the `.log' files are readable, and worse, they don't ensure that the
+## `.trs' files even exist.
+ redo_bases=`for i in $$bases; do \
+ am__f_ok $$i.trs && am__f_ok $$i.log || echo $$i; \
+ done`; \
+ if test -n "$$redo_bases"; then \
+## Uh-oh, either some `.log' files were unreadable, or some `.trs' files
+## were missing (or unreadable). We need to re-run the corresponding
+## tests in order to re-create them.
+ redo_logs=`for i in $$redo_bases; do echo $$i.log; done`; \
+ redo_results=`for i in $$redo_bases; do echo $$i.trs; done`; \
+## The exported variables are needed by the helper hook.
+ redo_bases="$$redo_bases" \
+ redo_logs="$$redo_logs" \
+ redo_results="$$redo_results" \
+ $(MAKE) $(AM_MAKEFLAGS) am--redo-logs || exit 1; \
+ else :; fi;
+## We need a new subshell to work portably with "make -n", since the
+## previous part of the recipe contained a $(MAKE) invocation.
+ @$(am__sh_e_setup); $(am__tty_colors); $(am__set_TESTS_bases); \
+ ws='[ ]'; \
+## List of test result files.
+ results=`for b in $$bases; do echo $$b.trs; done`; \
+ test -n "$$results" || results=/dev/null; \
+## Prepare data for the test suite summary. These do not take into account
+## unreadable test results, but they'll be appropriately updated later if
+## needed.
+ all=` grep "^$$ws*:test-result:" $$results | wc -l`; \
+ pass=` grep "^$$ws*:test-result:$$ws*PASS" $$results | wc -l`; \
+ fail=` grep "^$$ws*:test-result:$$ws*FAIL" $$results | wc -l`; \
+ skip=` grep "^$$ws*:test-result:$$ws*SKIP" $$results | wc -l`; \
+ xfail=`grep "^$$ws*:test-result:$$ws*XFAIL" $$results | wc -l`; \
+ xpass=`grep "^$$ws*:test-result:$$ws*XPASS" $$results | wc -l`; \
+ error=`grep "^$$ws*:test-result:$$ws*ERROR" $$results | wc -l`; \
+## Whether the testsuite was successful or not.
+ if test `expr $$fail + $$xpass + $$error` -eq 0; then \
+ success=true; \
+ else \
+ success=false; \
+ fi; \
+## Make $br a line of exactly 76 `=' characters, that will be used to
+## enclose the testsuite summary report when displayed on the console.
+ br='==================='; br=$$br$$br$$br$$br; \
+## When writing the test summary to the console, we want to color a line
+## reporting the count of some result *only* if at least one test
+## experienced such a result. This function is handy in this regard.
+ result_count () \
+ { \
+ if test x"$$1" = x"--maybe-color"; then \
+ maybe_colorize=yes; \
+ elif test x"$$1" = x"--no-color"; then \
+ maybe_colorize=no; \
+ else \
+ echo "$@: invalid 'result_count' usage" >&2; exit 4; \
+ fi; \
+ shift; \
+ desc=$$1 count=$$2; \
+ if test $$maybe_colorize = yes && test $$count -gt 0; then \
+ color_start=$$3 color_end=$$std; \
+ else \
+ color_start= color_end=; \
+ fi; \
+ echo "$${color_start}# $$desc $$count$${color_end}"; \
+ }; \
+## A shell function that creates the testsuite summary. We need it
+## because we have to create *two* summaries, one for test-suite.log,
+## and a possibly-colorized one for console output.
+ create_testsuite_report () \
+ { \
+ result_count $$1 "TOTAL:" $$all "$$brg"; \
+ result_count $$1 "PASS: " $$pass "$$grn"; \
+ result_count $$1 "SKIP: " $$skip "$$blu"; \
+ result_count $$1 "XFAIL:" $$xfail "$$lgn"; \
+ result_count $$1 "FAIL: " $$fail "$$red"; \
+ result_count $$1 "XPASS:" $$xpass "$$red"; \
+ result_count $$1 "ERROR:" $$error "$$mgn"; \
+ }; \
+## Write "global" testsuite log.
{ \
echo "$(PACKAGE_STRING): $(subdir)/$(TEST_SUITE_LOG)" | \
$(am__rst_title); \
- echo "$$msg"; \
+ create_testsuite_report --no-color; \
echo; \
echo ".. contents:: :depth: 2"; \
echo; \
- for f in $$list; do \
- test -r $$f && read line < $$f || line=; \
- case $$line in \
- PASS:*|XFAIL:*);; \
- *) echo; cat $$f;; \
- esac; \
- done; \
+ for i in $$bases; do \
+## FIXME: one fork per test -- this is horrendously inefficient!
+ if grep "^$$ws*:copy-in-global-log:$$ws*no$$ws*$$" $$i.trs \
+ >/dev/null; then continue; \
+ fi; \
+## Get the declared "global result" of the test.
+## FIXME: yet another one fork per test here!
+ glob_res=`sed -n -e "s/$$ws*$$//" \
+ -e "s/^$$ws*:global-test-result:$$ws*//p" \
+ $$i.trs`; \
+## If no global result is explicitly declared, we'll merely mark the
+## test as "RUN" in the global test log.
+ test -n "$$glob_res" || glob_res=RUN; \
+## Write the name and result of the test as an RST section title.
+ echo "$$glob_res: $$i" | $(am__rst_section); \
+## If we should have remade any unreadable `.log', above.
+ if test ! -r $$i.log; then \
+ echo "fatal: making $@: $$i.log is unreadable" >&2; \
+ exit 1; \
+ fi; \
+ cat $$i.log; echo; \
+ done; \
} >$(TEST_SUITE_LOG).tmp; \
mv $(TEST_SUITE_LOG).tmp $(TEST_SUITE_LOG); \
- if test "$$failures" -ne 0; then \
- msg="$${msg}See $(subdir)/$(TEST_SUITE_LOG). "; \
- if test -n "$(PACKAGE_BUGREPORT)"; then \
- msg="$${msg}Please report to $(PACKAGE_BUGREPORT). "; \
- fi; \
- fi; \
- test x"$$VERBOSE" = x || $$exit || cat $(TEST_SUITE_LOG); \
- $(am__tty_colors); \
- if $$exit; then \
+## Emit the test summary on the console.
+ if $$success; then \
col="$$grn"; \
else \
col="$$red"; \
+ test x"$$VERBOSE" = x || cat $(TEST_SUITE_LOG); \
fi; \
- echo "$$msg" | $(am__text_box) "col=$$col" "std=$$std"; \
-## The use of "exit 1" below is required to work around a FreeBSD make bug
-## (present only when running in concurrent mode). See automake bug#9245:
+## Multi line coloring is problematic with "less -R", so we really need
+## to color each line individually.
+ echo "$${col}$$br$${std}"; \
+ echo "$${col}Testsuite summary for $(PACKAGE_STRING)$${std}"; \
+ echo "$${col}$$br$${std}"; \
+## This is expected to go to the console, so it might have to be colorized.
+ create_testsuite_report --maybe-color; \
+ echo "$$col$$br$$std"; \
+ if $$success; then :; else \
+ echo "$${col}See $(subdir)/$(TEST_SUITE_LOG)$${std}"; \
+ if test -n "$(PACKAGE_BUGREPORT)"; then \
+ echo "$${col}Please report to $(PACKAGE_BUGREPORT)$${std}"; \
+ fi; \
+ echo "$$col$$br$$std"; \
+ fi; \
+## Be sure to exit with the proper exit status. The use of "exit 1" below
+## is required to work around a FreeBSD make bug (present only when running
+## in concurrent mode). See automake bug#9245:
## <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=9245>
## and FreeBSD PR bin/159730:
## <http://www.freebsd.org/cgi/query-pr.cgi?pr=159730>.
- $$exit || exit 1
+ $$success || exit 1
RECHECK_LOGS = $(TEST_LOGS)
check-TESTS:
## Expand $(RECHECK_LOGS) only once, to avoid exceeding line length limits.
@list='$(RECHECK_LOGS)'; test -z "$$list" || rm -f $$list
+ @list='$(RECHECK_LOGS:.log=.trs)'; test -z "$$list" || rm -f $$list
## We always have to remove TEST_SUITE_LOG, to ensure its rule is run
## in any case even in lazy mode: otherwise, if no test needs rerunning,
## or a prior run plus reruns all happen within the same timestamp
## 3.80 to erroneously expand $(TESTS_LOGS) to `foo.log .log'.
## Work around this bug.
@test -z "$(TEST_SUITE_LOG)" || rm -f $(TEST_SUITE_LOG)
- @list='$(TEST_LOGS)'; \
- list=`for f in $$list; do \
- test .log = $$f || echo $$f; \
- done | tr '\012\015' ' '`; \
+ @list='$(TEST_LOGS)'; \
+ list=`for i in $$list; do \
+ test .log = $$i || echo $$i; \
+ done | tr '\012\015' ' '`; \
+## This apparently useless munging helps to avoid a nasty bug (a
+## segmentation fault!) on Solaris XPG4 make.
+ list=`echo "$$list" | sed 's/ *$$//'`; \
$(MAKE) $(AM_MAKEFLAGS) $(TEST_SUITE_LOG) TEST_LOGS="$$list"
AM_RECURSIVE_TARGETS += check
## Rechecking failures. ##
## -------------------- ##
-## Rerun all FAILed or XPASSed tests.
+## Rerun all tests that experienced an error or an unexpected failure.
recheck recheck-html:
- @target=`echo $@ | sed 's,^re,,'`; \
- list='$(TEST_LOGS)'; \
- list=`for f in $$list; do \
- test -f $$f || continue; \
- if test -r $$f && read line < $$f; then \
- case $$line in FAIL*|XPASS*) echo $$f;; esac; \
- else echo $$f; fi; \
+ @ws='[ ]'; \
+ target=`echo $@ | sed 's,^re,,'`; \
+ $(am__set_TESTS_bases); \
+ list=`for i in $$bases; do \
+## Skip tests that haven't been run, but recover gracefully from deleted
+## `.trs' files.
+ test -f $$i.trs || test -f $$i.log || continue; \
+## FIXME: one fork per test -- this is horrendously inefficient!
+ grep "^$$ws*:recheck:$$ws*no$$ws*$$" $$i.trs \
+ >/dev/null 2>&1 || echo $$i.log; \
done | tr '\012\015' ' '`; \
## This apparently useless munging helps to avoid a nasty bug (a
## segmentation fault!) on Solaris XPG4 make.
## why we also try `dir='
elif test -f $$tst; then dir=; \
else dir="$(srcdir)/"; fi; \
- if $(TESTS_ENVIRONMENT) $${dir}$$tst; then \
+ if $(TESTS_ENVIRONMENT) $${dir}$$tst $(AM_TESTS_FD_REDIRECT); then \
## Success
all=`expr $$all + 1`; \
case " $(XFAIL_TESTS) " in \
## automake - create Makefile.in from Makefile.am
-## Copyright (C) 2008, 2009 Free Software Foundation, Inc.
+## Copyright (C) 2008, 2009, 2011 Free Software Foundation, Inc.
## This program is free software; you can redistribute it and/or modify
## it under the terms of the GNU General Public License as published by
## You should have received a copy of the GNU General Public License
## along with this program. If not, see <http://www.gnu.org/licenses/>.
-## From a test file to a log file.
+## From a test file to a .log and .trs file.
?GENERIC?%EXT%.log:
?!GENERIC?%OBJ%: %SOURCE%
- @p='%SOURCE%'; $(am__check_pre) %COMPILE% "$$tst" $(am__check_post)
+ @p='%SOURCE%'; $(am__check_pre) %DRIVER% --test-name "$$f" \
+ --log-file '%BASE%.log' --trs-file '%BASE%.trs' \
+ $(am__common_driver_flags) %DRIVER_FLAGS% -- %COMPILE% "$$tst" \
+ $(AM_TESTS_FD_REDIRECT)
## If no programs are built in this package, then this rule is removed
## at automake time. Otherwise, %am__EXEEXT% expands to a configure time
## conflict with the previous one.
if %am__EXEEXT%
?GENERIC?%EXT%$(EXEEXT).log:
- @p='%SOURCE%'; $(am__check_pre) %COMPILE% "$$tst" $(am__check_post)
+ @p='%SOURCE%'; $(am__check_pre) %DRIVER% --test-name "$$f" \
+ --log-file '%BASE%.log' --trs-file '%BASE%.trs' \
+ $(am__common_driver_flags) %DRIVER_FLAGS% -- %COMPILE% "$$tst" \
+ $(AM_TESTS_FD_REDIRECT)
endif %am__EXEEXT%
--- /dev/null
+#! /usr/bin/env perl
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# As a special exception to the GNU General Public License, if you
+# distribute this file as part of a program that contains a
+# configuration script generated by Autoconf, you may include it under
+# the same distribution terms that you use for the rest of that program.
+
+# This file is maintained in Automake, please report
+# bugs to <bug-automake@gnu.org> or send patches to
+# <automake-patches@gnu.org>.
+
+# ---------------------------------- #
+# Imports, static data, and setup. #
+# ---------------------------------- #
+
+use warnings FATAL => 'all';
+use strict;
+use Getopt::Long ();
+use TAP::Parser;
+
+my $VERSION = '2011-08-25.10'; # UTC
+
+my $ME = "tap-driver.pl";
+
+my $USAGE = <<'END';
+Usage:
+ tap-driver --test-name=NAME --log-file=PATH --trs-file=PATH
+ [--expect-failure={yes|no}] [--color-tests={yes|no}]
+ [--enable-hard-errors={yes|no}] [--ignore-exit]
+ [--diagnostic-string=STRING] [--merge|--no-merge]
+ [--comments|--no-comments] [--] TEST-COMMAND
+The `--test-name', `--log-file' and `--trs-file' options are mandatory.
+END
+
+my $HELP = "$ME: TAP-aware test driver for Automake testsuite harness." .
+ "\n" . $USAGE;
+
+# Keep this in sync with `lib/am/check.am:$(am__tty_colors)'.
+my %COLOR = (
+ red => "\e[0;31m",
+ grn => "\e[0;32m",
+ lgn => "\e[1;32m",
+ blu => "\e[1;34m",
+ mgn => "\e[0;35m",
+ brg => "\e[1m",
+ std => "\e[m",
+);
+
+# It's important that NO_PLAN evaluates "false" as a boolean.
+use constant NO_PLAN => 0;
+use constant EARLY_PLAN => 1;
+use constant LATE_PLAN => 2;
+
+# ------------------- #
+# Global variables. #
+# ------------------- #
+
+my $testno = 0; # Number of test results seen so far.
+my $bailed_out = 0; # Whether a "Bail out!" directive has been seen.
+my $parser; # TAP parser object (will be initialized later).
+
+# Whether the TAP plan has been seen or not, and if yes, which kind
+# it is ("early" is seen before any test result, "late" otherwise).
+my $plan_seen = NO_PLAN;
+
+# ----------------- #
+# Option parsing. #
+# ----------------- #
+
+my %cfg = (
+ "color-tests" => 0,
+ "expect-failure" => 0,
+ "merge" => 0,
+ "comments" => 0,
+ "ignore-exit" => 0,
+);
+
+my $test_script_name = undef;
+my $log_file = undef;
+my $trs_file = undef;
+my $diag_string = "#";
+
+Getopt::Long::GetOptions
+ (
+ 'help' => sub { print $HELP; exit 0; },
+ 'version' => sub { print "$ME $VERSION\n"; exit 0; },
+ 'test-name=s' => \$test_script_name,
+ 'log-file=s' => \$log_file,
+ 'trs-file=s' => \$trs_file,
+ 'color-tests=s' => \&bool_opt,
+ 'expect-failure=s' => \&bool_opt,
+ 'enable-hard-errors=s' => sub {}, # No-op.
+ 'diagnostic-string=s' => \$diag_string,
+ 'comments' => sub { $cfg{"comments"} = 1; },
+ 'no-comments' => sub { $cfg{"comments"} = 0; },
+ 'merge' => sub { $cfg{"merge"} = 1; },
+ 'no-merge' => sub { $cfg{"merge"} = 0; },
+ 'ignore-exit' => sub { $cfg{"ignore-exit"} = 1; },
+ ) or exit 1;
+
+# ------------- #
+# Prototypes. #
+# ------------- #
+
+sub add_test_result ($);
+sub bool_opt ($$);
+sub colored ($$);
+sub copy_in_global_log ();
+sub decorate_result ($);
+sub extract_tap_comment ($);
+sub get_global_test_result ();
+sub get_test_exit_message ();
+sub get_test_results ();
+sub handle_tap_bailout ($);
+sub handle_tap_plan ($);
+sub handle_tap_result ($);
+sub is_null_string ($);
+sub main (@);
+sub must_recheck ();
+sub report ($;$);
+sub start (@);
+sub stringify_result_obj ($);
+sub testsuite_error ($);
+sub trap_perl_warnings_and_errors ();
+sub write_test_results ();
+sub yn ($);
+
+# -------------- #
+# Subroutines. #
+# -------------- #
+
+sub bool_opt ($$)
+{
+ my ($opt, $val) = @_;
+ if ($val =~ /^(?:y|yes)\z/i)
+ {
+ $cfg{$opt} = 1;
+ }
+ elsif ($val =~ /^(?:n|no)\z/i)
+ {
+ $cfg{$opt} = 0;
+ }
+ else
+ {
+ die "$ME: invalid argument '$val' for option '$opt'\n";
+ }
+}
+
+# If the given string is undefined or empty, return true, otherwise
+# return false. This function is useful to avoid pitfalls like:
+# if ($message) { print "$message\n"; }
+# which wouldn't print anything if $message is the literal "0".
+sub is_null_string ($)
+{
+ my $str = shift;
+ return ! (defined $str and length $str);
+}
+
+# Convert a boolean to a "yes"/"no" string.
+sub yn ($)
+{
+ my $bool = shift;
+ return $bool ? "yes" : "no";
+}
+
+TEST_RESULTS :
+{
+ my (@test_results_list, %test_results_seen);
+
+ sub add_test_result ($)
+ {
+ my $res = shift;
+ push @test_results_list, $res;
+ $test_results_seen{$res} = 1;
+ }
+
+ sub get_test_results ()
+ {
+ return @test_results_list;
+ }
+
+ # Whether the test script should be re-run by "make recheck".
+ sub must_recheck ()
+ {
+ return grep { !/^(?:XFAIL|PASS|SKIP)$/ } (keys %test_results_seen);
+ }
+
+ # Whether the content of the log file associated to this test should
+ # be copied into the "global" test-suite.log.
+ sub copy_in_global_log ()
+ {
+ return grep { not $_ eq "PASS" } (keys %test_results_seen);
+ }
+
+ # FIXME: this can certainly be improved ...
+ sub get_global_test_result ()
+ {
+ return "ERROR"
+ if $test_results_seen{"ERROR"};
+ return "FAIL"
+ if $test_results_seen{"FAIL"} || $test_results_seen{"XPASS"};
+ return "SKIP"
+ if scalar keys %test_results_seen == 1 && $test_results_seen{"SKIP"};
+ return "PASS";
+ }
+
+}
+
+sub write_test_results ()
+{
+ open RES, ">", $trs_file or die "$ME: opening $trs_file: $!\n";
+ print RES ":global-test-result: " . get_global_test_result . "\n";
+ print RES ":recheck: " . yn (must_recheck) . "\n";
+ print RES ":copy-in-global-log: " . yn (copy_in_global_log) . "\n";
+ foreach my $result (get_test_results)
+ {
+ print RES ":test-result: $result\n";
+ }
+ close RES or die "$ME: closing $trs_file: $!\n";
+}
+
+sub trap_perl_warnings_and_errors ()
+{
+ $SIG{__WARN__} = $SIG{__DIE__} = sub
+ {
+ # Be sure to send the warning/error message to the original stderr
+ # (presumably the console), not into the log file.
+ open STDERR, ">&", \*OLDERR;
+ die @_;
+ }
+}
+
+sub start (@)
+{
+ # Redirect stderr and stdout to a temporary log file. Save the
+ # original stdout stream, since we need it to print testsuite
+ # progress output. Save original stderr stream, so that we can
+ # redirect warning and error messages from perl there.
+ open LOG, ">", $log_file or die "$ME: opening $log_file: $!\n";
+ open OLDOUT, ">&STDOUT" or die "$ME: duplicating stdout: $!\n";
+ open OLDERR, ">&STDERR" or die "$ME: duplicating stdout: $!\n";
+ trap_perl_warnings_and_errors;
+ open STDOUT, ">&LOG" or die "$ME: redirecting stdout: $!\n";
+ open STDERR, ">&LOG" or die "$ME: redirecting stderr: $!\n";
+ $parser = TAP::Parser->new ({ exec => \@_, merge => $cfg{merge} });
+ $parser->ignore_exit(1) if $cfg{"ignore-exit"};
+}
+
+sub get_test_exit_message ()
+{
+ my $wstatus = $parser->wait;
+ # Watch out for possible internal errors.
+ die "$ME: couldn't get the exit ststus of the TAP producer"
+ unless defined $wstatus;
+ # Return an undefined value if the producer exited with success.
+ return unless $wstatus;
+ # Otherwise, determine whether it exited with error or was terminated
+ # by a signal.
+ use POSIX qw (WIFEXITED WEXITSTATUS WIFSIGNALED WTERMSIG);
+ if (WIFEXITED ($wstatus))
+ {
+ return sprintf "exited with status %d", WEXITSTATUS ($wstatus);
+ }
+ elsif (WIFSIGNALED ($wstatus))
+ {
+ return sprintf "terminated by signal %d", WTERMSIG ($wstatus);
+ }
+ else
+ {
+ return "terminated abnormally";
+ }
+}
+
+sub stringify_result_obj ($)
+{
+ my $result_obj = shift;
+ my $COOKED_PASS = $cfg{"expect-failure"} ? "XPASS": "PASS";
+ my $COOKED_FAIL = $cfg{"expect-failure"} ? "XFAIL": "FAIL";
+ if ($result_obj->is_unplanned || $result_obj->number != $testno)
+ {
+ return "ERROR";
+ }
+ elsif ($plan_seen == LATE_PLAN)
+ {
+ return "ERROR";
+ }
+ elsif (!$result_obj->directive)
+ {
+ return $result_obj->is_ok ? $COOKED_PASS: $COOKED_FAIL;
+ }
+ elsif ($result_obj->has_todo)
+ {
+ return $result_obj->is_actual_ok ? "XPASS" : "XFAIL";
+ }
+ elsif ($result_obj->has_skip)
+ {
+ return $result_obj->is_ok ? "SKIP" : $COOKED_FAIL;
+ }
+ die "$ME: INTERNAL ERROR"; # NOTREACHED
+}
+
+sub colored ($$)
+{
+ my ($color_name, $text) = @_;
+ return $COLOR{$color_name} . $text . $COLOR{'std'};
+}
+
+sub decorate_result ($)
+{
+ my $result = shift;
+ return $result unless $cfg{"color-tests"};
+ my %color_for_result =
+ (
+ "ERROR" => 'mgn',
+ "PASS" => 'grn',
+ "XPASS" => 'red',
+ "FAIL" => 'red',
+ "XFAIL" => 'lgn',
+ "SKIP" => 'blu',
+ );
+ if (my $color = $color_for_result{$result})
+ {
+ return colored ($color, $result);
+ }
+ else
+ {
+ return $result; # Don't colorize unknown stuff.
+ }
+}
+
+sub report ($;$)
+{
+ my ($msg, $result, $explanation) = (undef, @_);
+ if ($result =~ /^(?:X?(?:PASS|FAIL)|SKIP|ERROR)/)
+ {
+ $msg = ": $test_script_name";
+ add_test_result $result;
+ }
+ elsif ($result eq "#")
+ {
+ $msg = " $test_script_name:";
+ }
+ else
+ {
+ die "$ME: INTERNAL ERROR"; # NOTREACHED
+ }
+ $msg .= " $explanation" if defined $explanation;
+ $msg .= "\n";
+ # Output on console might be colorized.
+ print OLDOUT decorate_result ($result) . $msg;
+ # Log the result in the log file too, to help debugging (this is
+ # especially true when said result is a TAP error or "Bail out!").
+ print $result . $msg;
+}
+
+sub testsuite_error ($)
+{
+ report "ERROR", "- $_[0]";
+}
+
+sub handle_tap_result ($)
+{
+ $testno++;
+ my $result_obj = shift;
+
+ my $test_result = stringify_result_obj $result_obj;
+ my $string = $result_obj->number;
+
+ my $description = $result_obj->description;
+ $string .= " $description"
+ unless is_null_string $description;
+
+ if ($plan_seen == LATE_PLAN)
+ {
+ $string .= " # AFTER LATE PLAN";
+ }
+ elsif ($result_obj->is_unplanned)
+ {
+ $string .= " # UNPLANNED";
+ }
+ elsif ($result_obj->number != $testno)
+ {
+ $string .= " # OUT-OF-ORDER (expecting $testno)";
+ }
+ elsif (my $directive = $result_obj->directive)
+ {
+ $string .= " # $directive";
+ my $explanation = $result_obj->explanation;
+ $string .= " $explanation"
+ unless is_null_string $explanation;
+ }
+
+ report $test_result, $string;
+}
+
+sub handle_tap_plan ($)
+{
+ my $plan = shift;
+ if ($plan_seen)
+ {
+ # Error, only one plan per stream is acceptable.
+ testsuite_error "multiple test plans";
+ return;
+ }
+ # The TAP plan can come before or after *all* the TAP results; we speak
+ # respectively of an "early" or a "late" plan. If we see the plan line
+ # after at least one TAP result has been seen, assume we have a late
+ # plan; in this case, any further test result seen after the plan will
+ # be flagged as an error.
+ $plan_seen = ($testno >= 1 ? LATE_PLAN : EARLY_PLAN);
+ # If $testno > 0, we have an error ("too many tests run") that will be
+ # automatically dealt with later, so don't worry about it here. If
+ # $plan_seen is true, we have an error due to a repeated plan, and that
+ # has already been dealt with above. Otherwise, we have a valid "plan
+ # with SKIP" specification, and should report it as a particular kind
+ # of SKIP result.
+ if ($plan->directive && $testno == 0)
+ {
+ my $explanation = is_null_string ($plan->explanation) ?
+ undef : "- " . $plan->explanation;
+ report "SKIP", $explanation;
+ }
+}
+
+sub handle_tap_bailout ($)
+{
+ my ($bailout, $msg) = ($_[0], "Bail out!");
+ $bailed_out = 1;
+ $msg .= " " . $bailout->explanation
+ unless is_null_string $bailout->explanation;
+ testsuite_error $msg;
+}
+
+sub extract_tap_comment ($)
+{
+ my $line = shift;
+ if (index ($line, $diag_string) == 0)
+ {
+ # Strip leading `$diag_string' from `$line'.
+ $line = substr ($line, length ($diag_string));
+ # And strip any leading and trailing whitespace left.
+ $line =~ s/(?:^\s*|\s*$)//g;
+ # Return what is left (if any).
+ return $line;
+ }
+ return "";
+}
+
+sub main (@)
+{
+ start @_;
+
+ while (defined (my $cur = $parser->next))
+ {
+ # Verbatim copy any input line into the log file.
+ print $cur->raw . "\n";
+ # Parsing of TAP input should stop after a "Bail out!" directive.
+ next if $bailed_out;
+
+ if ($cur->is_plan)
+ {
+ handle_tap_plan ($cur);
+ }
+ elsif ($cur->is_test)
+ {
+ handle_tap_result ($cur);
+ }
+ elsif ($cur->is_bailout)
+ {
+ handle_tap_bailout ($cur);
+ }
+ elsif ($cfg{comments})
+ {
+ my $comment = extract_tap_comment ($cur->raw);
+ report "#", "$comment" if length $comment;
+ }
+ }
+ # A "Bail out!" directive should cause us to ignore any following TAP
+ # error, as well as a non-zero exit status from the TAP producer.
+ if (!$bailed_out)
+ {
+ if (!$plan_seen)
+ {
+ testsuite_error "missing test plan";
+ }
+ elsif ($parser->tests_planned != $parser->tests_run)
+ {
+ my ($planned, $run) = ($parser->tests_planned, $parser->tests_run);
+ my $bad_amount = $run > $planned ? "many" : "few";
+ testsuite_error (sprintf "too %s tests run (expected %d, got %d)",
+ $bad_amount, $planned, $run);
+ }
+ if (!$cfg{"ignore-exit"})
+ {
+ my $msg = get_test_exit_message ();
+ testsuite_error $msg if $msg;
+ }
+ }
+ write_test_results;
+ close LOG or die "$ME: closing $log_file: $!\n";
+ exit 0;
+}
+
+# ----------- #
+# Main code. #
+# ----------- #
+
+main @ARGV;
+
+# Local Variables:
+# perl-indent-level: 2
+# perl-continued-statement-offset: 2
+# perl-continued-brace-offset: 0
+# perl-brace-offset: 0
+# perl-brace-imaginary-offset: 0
+# perl-label-offset: -2
+# cperl-indent-level: 2
+# cperl-brace-offset: 0
+# cperl-continued-brace-offset: 0
+# cperl-label-offset: -2
+# cperl-extra-newline-before-brace: t
+# cperl-merge-trailing-else: nil
+# cperl-continued-statement-offset: 2
+# eval: (add-hook 'write-file-hooks 'time-stamp)
+# time-stamp-start: "my $VERSION = "
+# time-stamp-format: "'%:y-%02m-%02d.%02H'"
+# time-stamp-time-zone: "UTC"
+# time-stamp-end: "; # UTC"
+# End:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# As a special exception to the GNU General Public License, if you
+# distribute this file as part of a program that contains a
+# configuration script generated by Autoconf, you may include it under
+# the same distribution terms that you use for the rest of that program.
+
+# This file is maintained in Automake, please report
+# bugs to <bug-automake@gnu.org> or send patches to
+# <automake-patches@gnu.org>.
+
+scriptversion=2011-08-25.11; # UTC
+
+# Make unconditional expansion of undefined variables an error. This
+# helps a lot in preventing typo-related bugs.
+set -u
+
+me=tap-driver.sh
+
+fatal ()
+{
+ echo "$me: fatal: $*" >&2
+ exit 1
+}
+
+usage_error ()
+{
+ echo "$me: $*" >&2
+ print_usage >&2
+ exit 2
+}
+
+print_usage ()
+{
+ cat <<END
+Usage:
+ tap-driver.sh --test-name=NAME --log-file=PATH --trs-file=PATH
+ [--expect-failure={yes|no}] [--color-tests={yes|no}]
+ [--enable-hard-errors={yes|no}] [--ignore-exit]
+ [--diagnostic-string=STRING] [--merge|--no-merge]
+ [--comments|--no-comments] [--] TEST-COMMAND
+The \`--test-name', \`--log-file' and \`--trs-file' options are mandatory.
+END
+}
+
+# TODO: better error handling in option parsing (in particular, ensure
+# TODO: $log_file, $trs_file and $test_name are defined).
+test_name= # Used for reporting.
+log_file= # Where to save the result and output of the test script.
+trs_file= # Where to save the metadata of the test run.
+expect_failure=0
+color_tests=0
+merge=0
+ignore_exit=0
+comments=0
+diag_string='#'
+while test $# -gt 0; do
+ case $1 in
+ --help) print_usage; exit $?;;
+ --version) echo "$me $scriptversion"; exit $?;;
+ --test-name) test_name=$2; shift;;
+ --log-file) log_file=$2; shift;;
+ --trs-file) trs_file=$2; shift;;
+ --color-tests) color_tests=$2; shift;;
+ --expect-failure) expect_failure=$2; shift;;
+ --enable-hard-errors) shift;; # No-op.
+ --merge) merge=1;;
+ --no-merge) merge=0;;
+ --ignore-exit) ignore_exit=1;;
+ --comments) comments=1;;
+ --no-comments) comments=0;;
+ --diagnostic-string) diag_string=$2; shift;;
+ --) shift; break;;
+ -*) usage_error "invalid option: '$1'";;
+ esac
+ shift
+done
+
+test $# -gt 0 || usage_error "missing test command"
+
+case $expect_failure in
+ yes) expect_failure=1;;
+ *) expect_failure=0;;
+esac
+
+if test $color_tests = yes; then
+ init_colors='
+ color_map["red"]="\e[0;31m" # Red.
+ color_map["grn"]="\e[0;32m" # Green.
+ color_map["lgn"]="\e[1;32m" # Light green.
+ color_map["blu"]="\e[1;34m" # Blue.
+ color_map["mgn"]="\e[0;35m" # Magenta.
+ color_map["std"]="\e[m" # No color.
+ color_for_result["ERROR"] = "mgn"
+ color_for_result["PASS"] = "grn"
+ color_for_result["XPASS"] = "red"
+ color_for_result["FAIL"] = "red"
+ color_for_result["XFAIL"] = "lgn"
+ color_for_result["SKIP"] = "blu"'
+else
+ init_colors=''
+fi
+
+{
+ { if test $merge -gt 0; then
+ exec 2>&1
+ else
+ exec 2>&3
+ fi
+ "$@"
+ echo $?
+ } | LC_ALL=C ${AM_TAP_AWK-awk} \
+ -v me="$me" \
+ -v test_script_name="$test_name" \
+ -v log_file="$log_file" \
+ -v trs_file="$trs_file" \
+ -v expect_failure="$expect_failure" \
+ -v merge="$merge" \
+ -v ignore_exit="$ignore_exit" \
+ -v comments="$comments" \
+ -v diag_string="$diag_string" \
+'
+# FIXME: the usages of "cat >&3" below could be optimized when using
+# FIXME: GNU awk, and/on on systems that supports /dev/fd/.
+
+# Implementation note: in what follows, `result_obj` will be an
+# associative array that (partly) simulates a TAP result object
+# from the `TAP::Parser` perl module.
+
+## ----------- ##
+## FUNCTIONS ##
+## ----------- ##
+
+function fatal(msg)
+{
+ print me ": " msg | "cat >&2"
+ exit 1
+}
+
+function abort(where)
+{
+ fatal("internal error " where)
+}
+
+# Convert a boolean to a "yes"/"no" string.
+function yn(bool)
+{
+ return bool ? "yes" : "no";
+}
+
+function add_test_result(result)
+{
+ if (!test_results_index)
+ test_results_index = 0
+ test_results_list[test_results_index] = result
+ test_results_index += 1
+ test_results_seen[result] = 1;
+}
+
+# Whether the test script should be re-run by "make recheck".
+function must_recheck()
+{
+ for (k in test_results_seen)
+ if (k != "XFAIL" && k != "PASS" && k != "SKIP")
+ return 1
+ return 0
+}
+
+# Whether the content of the log file associated to this test should
+# be copied into the "global" test-suite.log.
+function copy_in_global_log()
+{
+ for (k in test_results_seen)
+ if (k != "PASS")
+ return 1
+ return 0
+}
+
+# FIXME: this can certainly be improved ...
+function get_global_test_result()
+{
+ if ("ERROR" in test_results_seen)
+ return "ERROR"
+ if ("FAIL" in test_results_seen || "XPASS" in test_results_seen)
+ return "FAIL"
+ all_skipped = 1
+ for (k in test_results_seen)
+ if (k != "SKIP")
+ all_skipped = 0
+ if (all_skipped)
+ return "SKIP"
+ return "PASS";
+}
+
+function stringify_result_obj(result_obj)
+{
+ if (result_obj["is_unplanned"] || result_obj["number"] != testno)
+ return "ERROR"
+
+ if (plan_seen == LATE_PLAN)
+ return "ERROR"
+
+ if (result_obj["directive"] == "TODO")
+ return result_obj["is_ok"] ? "XPASS" : "XFAIL"
+
+ if (result_obj["directive"] == "SKIP")
+ return result_obj["is_ok"] ? "SKIP" : COOKED_FAIL;
+
+ if (length(result_obj["directive"]))
+ abort("in function stringify_result_obj()")
+
+ return result_obj["is_ok"] ? COOKED_PASS : COOKED_FAIL
+}
+
+function decorate_result(result)
+{
+ color_name = color_for_result[result]
+ if (color_name)
+ return color_map[color_name] "" result "" color_map["std"]
+ # If we are not using colorized output, or if we do not know how
+ # to colorize the given result, we should return it unchanged.
+ return result
+}
+
+function report(result, details)
+{
+ if (result ~ /^(X?(PASS|FAIL)|SKIP|ERROR)/)
+ {
+ msg = ": " test_script_name
+ add_test_result(result)
+ }
+ else if (result == "#")
+ {
+ msg = " " test_script_name ":"
+ }
+ else
+ {
+ abort("in function report()")
+ }
+ if (length(details))
+ msg = msg " " details
+ # Output on console might be colorized.
+ print decorate_result(result) msg
+ # Log the result in the log file too, to help debugging (this is
+ # especially true when said result is a TAP error or "Bail out!").
+ print result msg | "cat >&3";
+}
+
+function testsuite_error(error_message)
+{
+ report("ERROR", "- " error_message)
+}
+
+function handle_tap_result()
+{
+ details = result_obj["number"];
+ if (length(result_obj["description"]))
+ details = details " " result_obj["description"]
+
+ if (plan_seen == LATE_PLAN)
+ {
+ details = details " # AFTER LATE PLAN";
+ }
+ else if (result_obj["is_unplanned"])
+ {
+ details = details " # UNPLANNED";
+ }
+ else if (result_obj["number"] != testno)
+ {
+ details = sprintf("%s # OUT-OF-ORDER (expecting %d)",
+ details, testno);
+ }
+ else if (result_obj["directive"])
+ {
+ details = details " # " result_obj["directive"];
+ if (length(result_obj["explanation"]))
+ details = details " " result_obj["explanation"]
+ }
+
+ report(stringify_result_obj(result_obj), details)
+}
+
+# `skip_reason` should be empty whenever planned > 0.
+function handle_tap_plan(planned, skip_reason)
+{
+ planned += 0 # Avoid getting confused if, say, `planned` is "00"
+ if (length(skip_reason) && planned > 0)
+ abort("in function handle_tap_plan()")
+ if (plan_seen)
+ {
+ # Error, only one plan per stream is acceptable.
+ testsuite_error("multiple test plans")
+ return;
+ }
+ planned_tests = planned
+ # The TAP plan can come before or after *all* the TAP results; we speak
+ # respectively of an "early" or a "late" plan. If we see the plan line
+ # after at least one TAP result has been seen, assume we have a late
+ # plan; in this case, any further test result seen after the plan will
+ # be flagged as an error.
+ plan_seen = (testno >= 1 ? LATE_PLAN : EARLY_PLAN)
+ # If testno > 0, we have an error ("too many tests run") that will be
+ # automatically dealt with later, so do not worry about it here. If
+ # $plan_seen is true, we have an error due to a repeated plan, and that
+ # has already been dealt with above. Otherwise, we have a valid "plan
+ # with SKIP" specification, and should report it as a particular kind
+ # of SKIP result.
+ if (planned == 0 && testno == 0)
+ {
+ if (length(skip_reason))
+ skip_reason = "- " skip_reason;
+ report("SKIP", skip_reason);
+ }
+}
+
+function extract_tap_comment(line)
+{
+ if (index(line, diag_string) == 1)
+ {
+ # Strip leading `diag_string` from `line`.
+ line = substr(line, length(diag_string) + 1)
+ # And strip any leading and trailing whitespace left.
+ sub("^[ \t]*", "", line)
+ sub("[ \t]*$", "", line)
+ # Return what is left (if any).
+ return line;
+ }
+ return "";
+}
+
+# When this function is called, we know that line is a TAP result line,
+# so that it matches the (perl) RE "^(not )?ok\b".
+function setup_result_obj(line)
+{
+ # Get the result, and remove it from the line.
+ result_obj["is_ok"] = (substr(line, 1, 2) == "ok" ? 1 : 0)
+ sub("^(not )?ok[ \t]*", "", line)
+
+ # If the result has an explicit number, get it and strip it; otherwise,
+ # automatically assing the next progresive number to it.
+ if (line ~ /^[0-9]+$/ || line ~ /^[0-9]+[^a-zA-Z0-9_]/)
+ {
+ match(line, "^[0-9]+")
+ # The final `+ 0` is to normalize numbers with leading zeros.
+ result_obj["number"] = substr(line, 1, RLENGTH) + 0
+ line = substr(line, RLENGTH + 1)
+ }
+ else
+ {
+ result_obj["number"] = testno
+ }
+
+ if (plan_seen == LATE_PLAN)
+ # No further test results are acceptable after a "late" TAP plan
+ # has been seen.
+ result_obj["is_unplanned"] = 1
+ else if (plan_seen && testno > planned_tests)
+ result_obj["is_unplanned"] = 1
+ else
+ result_obj["is_unplanned"] = 0
+
+ # Strip trailing and leading whitespace.
+ sub("^[ \t]*", "", line)
+ sub("[ \t]*$", "", line)
+
+ # This will have to be corrected if we have a "TODO"/"SKIP" directive.
+ result_obj["description"] = line
+ result_obj["directive"] = ""
+ result_obj["explanation"] = ""
+
+ if (index(line, "#") == 0)
+ return # No possible directive, nothing more to do.
+
+ # Directives are case-insensitive.
+ rx = "[ \t]*#[ \t]*([tT][oO][dD][oO]|[sS][kK][iI][pP])[ \t]*"
+
+ # See whether we have the directive, and if yes, where.
+ pos = match(line, rx "$")
+ if (!pos)
+ pos = match(line, rx "[^a-zA-Z0-9_]")
+
+ # If there was no TAP directive, we have nothing more to do.
+ if (!pos)
+ return
+
+ # Let`s now see if the TAP directive has been escaped. For example:
+ # escaped: ok \# SKIP
+ # not escaped: ok \\# SKIP
+ # escaped: ok \\\\\# SKIP
+ # not escaped: ok \ # SKIP
+ if (substr(line, pos, 1) == "#")
+ {
+ bslash_count = 0
+ for (i = pos; i > 1 && substr(line, i - 1, 1) == "\\"; i--)
+ bslash_count += 1
+ if (bslash_count % 2)
+ return # Directive was escaped.
+ }
+
+ # Strip the directive and its explanation (if any) from the test
+ # description.
+ result_obj["description"] = substr(line, 1, pos - 1)
+ # Now remove the test description from the line, that has been dealt
+ # with already.
+ line = substr(line, pos)
+ # Strip the directive, and save its value (normalized to upper case).
+ sub("^[ \t]*#[ \t]*", "", line)
+ result_obj["directive"] = toupper(substr(line, 1, 4))
+ line = substr(line, 5)
+ # Now get the explanation for the directive (if any), with leading
+ # and trailing whitespace removed.
+ sub("^[ \t]*", "", line)
+ sub("[ \t]*$", "", line)
+ result_obj["explanation"] = line
+}
+
+function get_test_exit_message(status)
+{
+ if (status == 0)
+ return ""
+ if (status !~ /^[1-9][0-9]*$/)
+ abort("getting exit status")
+ if (status < 127)
+ exit_details = ""
+ else if (status == 127)
+ exit_details = " (command not found?)"
+ else if (status >= 128 && status <= 255)
+ exit_details = sprintf(" (terminated by signal %d?)", status - 128)
+ else if (status >= 256)
+ exit_details = " (abnormal termination)"
+ return sprintf("exited with status %d%s", status, exit_details)
+}
+
+function write_test_results()
+{
+ print ":global-test-result: " get_global_test_result() > trs_file
+ print ":recheck: " yn(must_recheck()) > trs_file
+ print ":copy-in-global-log: " yn(copy_in_global_log()) > trs_file
+ for (i = 0; i < test_results_index; i += 1)
+ print ":test-result: " test_results_list[i] > trs_file
+ close(trs_file);
+}
+
+BEGIN {
+
+## ------- ##
+## SETUP ##
+## ------- ##
+
+'"$init_colors"'
+
+# Properly initialized once the TAP plan is seen.
+planned_tests = 0
+
+COOKED_PASS = expect_failure ? "XPASS": "PASS";
+COOKED_FAIL = expect_failure ? "XFAIL": "FAIL";
+
+# Enumeration-like constants to remember which kind of plan (if any)
+# has been seen. It is important that NO_PLAN evaluates "false" as
+# a boolean.
+NO_PLAN = 0
+EARLY_PLAN = 1
+LATE_PLAN = 2
+
+testno = 0 # Number of test results seen so far.
+bailed_out = 0 # Whether a "Bail out!" directive has been seen.
+
+# Whether the TAP plan has been seen or not, and if yes, which kind
+# it is ("early" is seen before any test result, "late" otherwise).
+plan_seen = NO_PLAN
+
+## --------- ##
+## PARSING ##
+## --------- ##
+
+is_first_read = 1
+
+while (1)
+ {
+ # Involutions required so that we are able to read the exit status
+ # from the last input line.
+ st = getline
+ if (st < 0) # I/O error.
+ fatal("I/O error while reading from input stream")
+ else if (st == 0) # End-of-input
+ {
+ if (is_first_read)
+ abort("in input loop: only one input line")
+ break
+ }
+ if (is_first_read)
+ {
+ is_first_read = 0
+ nextline = $0
+ continue
+ }
+ else
+ {
+ curline = nextline
+ nextline = $0
+ $0 = curline
+ }
+ # Copy any input line verbatim into the log file.
+ print | "cat >&3"
+ # Parsing of TAP input should stop after a "Bail out!" directive.
+ if (bailed_out)
+ continue
+
+ # TAP test result.
+ if ($0 ~ /^(not )?ok$/ || $0 ~ /^(not )?ok[^a-zA-Z0-9_]/)
+ {
+ testno += 1
+ setup_result_obj($0)
+ handle_tap_result()
+ }
+ # TAP plan (normal or "SKIP" without explanation).
+ else if ($0 ~ /^1\.\.[0-9]+[ \t]*$/)
+ {
+ # The next two lines will put the number of planned tests in $0.
+ sub("^1\\.\\.", "")
+ sub("[^0-9]*$", "")
+ handle_tap_plan($0, "")
+ continue
+ }
+ # TAP "SKIP" plan, with an explanation.
+ else if ($0 ~ /^1\.\.0+[ \t]*#/)
+ {
+ # The next lines will put the skip explanation in $0, stripping
+ # any leading and trailing whitespace. This is a little more
+ # tricky in truth, since we want to also strip a potential leading
+ # "SKIP" string from the message.
+ sub("^[^#]*#[ \t]*(SKIP[: \t][ \t]*)?", "")
+ sub("[ \t]*$", "");
+ handle_tap_plan(0, $0)
+ }
+ # "Bail out!" magic.
+ else if ($0 ~ /^Bail out!/)
+ {
+ bailed_out = 1
+ # Get the bailout message (if any), with leading and trailing
+ # whitespace stripped. The message remains stored in `$0`.
+ sub("^Bail out![ \t]*", "");
+ sub("[ \t]*$", "");
+ # Format the error message for the
+ bailout_message = "Bail out!"
+ if (length($0))
+ bailout_message = bailout_message " " $0
+ testsuite_error(bailout_message)
+ }
+ # Maybe we have too look for dianogtic comments too.
+ else if (comments != 0)
+ {
+ comment = extract_tap_comment($0);
+ if (length(comment))
+ report("#", comment);
+ }
+ }
+
+## -------- ##
+## FINISH ##
+## -------- ##
+
+# A "Bail out!" directive should cause us to ignore any following TAP
+# error, as well as a non-zero exit status from the TAP producer.
+if (!bailed_out)
+ {
+ if (!plan_seen)
+ {
+ testsuite_error("missing test plan")
+ }
+ else if (planned_tests != testno)
+ {
+ bad_amount = testno > planned_tests ? "many" : "few"
+ testsuite_error(sprintf("too %s tests run (expected %d, got %d)",
+ bad_amount, planned_tests, testno))
+ }
+ if (!ignore_exit)
+ {
+ # Fetch exit status from the last line.
+ exit_message = get_test_exit_message(nextline)
+ if (exit_message)
+ testsuite_error(exit_message)
+ }
+ }
+
+write_test_results()
+
+exit 0
+
+} # End of "BEGIN" block.
+'
+
+# TODO: document that we consume the file descriptor 3 :-(
+} 3>"$log_file"
+
+test $? -eq 0 || fatal "I/O or internal error"
+
+# Local Variables:
+# mode: shell-script
+# sh-indentation: 2
+# eval: (add-hook 'write-file-hooks 'time-stamp)
+# time-stamp-start: "scriptversion="
+# time-stamp-format: "%:y-%02m-%02d.%02H"
+# time-stamp-time-zone: "UTC"
+# time-stamp-end: "; # UTC"
+# End:
--- /dev/null
+#! /bin/sh
+# test-driver - basic driver script for the `parallel-tests' mode.
+
+scriptversion=2011-08-17.14; # UTC
+
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# As a special exception to the GNU General Public License, if you
+# distribute this file as part of a program that contains a
+# configuration script generated by Autoconf, you may include it under
+# the same distribution terms that you use for the rest of that program.
+
+# This file is maintained in Automake, please report
+# bugs to <bug-automake@gnu.org> or send patches to
+# <automake-patches@gnu.org>.
+
+# Make unconditional expansion of undefined variables an error. This
+# helps a lot in preventing typo-related bugs.
+set -u
+
+usage_error ()
+{
+ echo "$0: $*" >&2
+ print_usage >&2
+ exit 2
+}
+
+print_usage ()
+{
+ cat <<END
+Usage:
+ test-driver --test-name=NAME --log-file=PATH --trs-file=PATH
+ [--expect-failure={yes|no}] [--color-tests={yes|no}]
+ [--enable-hard-errors={yes|no}] [--] TEST-SCRIPT
+The \`--test-name', \`--log-file' and \`--trs-file' options are mandatory.
+END
+}
+
+# TODO: better error handling in option parsing (in particular, ensure
+# TODO: $log_file, $trs_file and $test_name are defined).
+test_name= # Used for reporting.
+log_file= # Where to save the output of the test script.
+trs_file= # Where to save the metadata of the test run.
+expect_failure=no
+color_tests=no
+enable_hard_errors=yes
+while test $# -gt 0; do
+ case $1 in
+ --help) print_usage; exit $?;;
+ --version) echo "test-driver $scriptversion"; exit $?;;
+ --test-name) test_name=$2; shift;;
+ --log-file) log_file=$2; shift;;
+ --trs-file) trs_file=$2; shift;;
+ --color-tests) color_tests=$2; shift;;
+ --expect-failure) expect_failure=$2; shift;;
+ --enable-hard-errors) enable_hard_errors=$2; shift;;
+ --) shift; break;;
+ -*) usage_error "invalid option: '$1'";;
+ esac
+ shift
+done
+
+if test $color_tests = yes; then
+ # Keep this in sync with `lib/am/check.am:$(am__tty_colors)'.
+ red='\e[0;31m' # Red.
+ grn='\e[0;32m' # Green.
+ lgn='\e[1;32m' # Light green.
+ blu='\e[1;34m' # Blue.
+ mgn='\e[0;35m' # Magenta.
+ std='\e[m' # No color.
+else
+ red= grn= lgn= blu= mgn= std=
+fi
+
+do_exit='rm -f $log_file $trs_file; (exit $st); exit $st'
+trap "st=129; $do_exit" 1
+trap "st=130; $do_exit" 2
+trap "st=141; $do_exit" 13
+trap "st=143; $do_exit" 15
+
+# Test script is run here.
+"$@" >$log_file 2>&1
+estatus=$?
+if test $enable_hard_errors = no && test $estatus -eq 99; then
+ estatus=1
+fi
+
+case $estatus:$expect_failure in
+ 0:yes) col=$red res=XPASS recheck=yes gcopy=yes;;
+ 0:*) col=$grn res=PASS recheck=no gcopy=no;;
+ 77:*) col=$blu res=SKIP recheck=no gcopy=yes;;
+ 99:*) col=$mgn res=ERROR recheck=yes gcopy=yes;;
+ *:yes) col=$lgn res=XFAIL recheck=no gcopy=yes;;
+ *:*) col=$red res=FAIL recheck=yes gcopy=yes;;
+esac
+
+# Report outcome to console.
+echo "${col}${res}${std}: $test_name"
+
+# Register the test result, and other relevant metadata.
+echo ":test-result: $res" > $trs_file
+echo ":global-test-result: $res" >> $trs_file
+echo ":recheck: $recheck" >> $trs_file
+echo ":copy-in-global-log: $gcopy" >> $trs_file
+
+# Local Variables:
+# mode: shell-script
+# sh-indentation: 2
+# eval: (add-hook 'write-file-hooks 'time-stamp)
+# time-stamp-start: "scriptversion="
+# time-stamp-format: "%:y-%02m-%02d.%02H"
+# time-stamp-time-zone: "UTC"
+# time-stamp-end: "; # UTC"
+# End:
PRE_UNINSTALL = :
POST_UNINSTALL = :
build_triplet = @build@
+host_triplet = @host@
subdir = m4
DIST_COMMON = $(dist_m4data_DATA) $(srcdir)/Makefile.am \
$(srcdir)/Makefile.in
docdir = @docdir@
dvidir = @dvidir@
exec_prefix = @exec_prefix@
+host = @host@
host_alias = @host_alias@
+host_cpu = @host_cpu@
+host_os = @host_os@
+host_vendor = @host_vendor@
htmldir = @htmldir@
includedir = @includedir@
infodir = @infodir@
/aclocal-*
/automake-*
/defs-static
-/instspc-tests.am
-/parallel-tests.am
+/wrap-tests.am
/*.dir
/*.log
-/*.log-t
-/*-p.test
-/instspc-*-build.test
-/instspc-*-install.test
-/*.test-t
+/*.trs
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-MAINTAINERCLEANFILES =
+# Run the tests with the shell detected at configure time.
+LOG_COMPILER = $(SHELL)
+
+TEST_EXTENSIONS = .test .tap
+TEST_LOG_COMPILER = $(LOG_COMPILER)
+TAP_LOG_COMPILER = $(LOG_COMPILER)
+
+TAP_LOG_DRIVER = AM_TAP_AWK='$(AWK)' $(SHELL) $(top_srcdir)/lib/tap-driver.sh
+
+AM_TAP_LOG_DRIVER_FLAGS = --merge --comments
+## Avoid spurious TAP diagnostic. We use an awful indirection with a
+## shell command substitution here, since the string needs to contain
+## `#' characters, and these can't be escaped portably in a make macro
+## definition.
+## FIXME: this is quite inefficient though, as it adds one extra fork
+## FIXME: per TAP test script; this is OK for the moment, since we have
+## FIXME: few such scripts, but might become an issue in the future.
+## Keep this in sync with the definition of $diag_string_ in tests/defs.
+AM_TAP_LOG_DRIVER_FLAGS += --diagnostic-string `printf '\043%%\043\n'`
+
EXTRA_DIST = ChangeLog-old
+## Will be updated later.
+TESTS =
+
XFAIL_TESTS = \
all.test \
-auxdir2.test \
cond17.test \
gcj6.test \
override-conditional-2.test \
yacc-dist-nobuild-subdir.test \
txinfo5.test
-
-include $(srcdir)/parallel-tests.am
-
-$(srcdir)/parallel-tests.am: gen-parallel-tests Makefile.am
- $(AM_V_GEN)($(am__cd) $(srcdir) && $(SHELL) ./gen-parallel-tests) >$@
-
-$(parallel_tests): Makefile.am
- $(AM_V_at)rm -f $@ $@-t
- $(AM_V_GEN)input=`echo $@ | sed 's,.*/,,; s,-p.test$$,.test,'`; \
- { echo '#!/bin/sh'; \
- echo '# DO NOT EDIT! GENERATED AUTOMATICALLY!'; \
- echo; \
- echo '# Ensure proper definition of $$testsrcdir.'; \
- echo '. ./defs-static || exit 99'; \
- echo 'test -n "$$testsrcdir" || exit 99 # sanity check'; \
- echo; \
- echo "# Run the test with Automake's parallel-tests driver enabled."; \
- echo 'parallel_tests=yes'; \
- echo "# In the spirit of VPATH, we prefer a test in the build tree"; \
- echo "# over one in the source tree."; \
- echo "if test -f \"./$$input\"; then"; \
- echo " . \"./$$input\""; \
- echo 'else'; \
- echo " . \"\$$testsrcdir/$$input\""; \
- echo 'fi'; \
- } > $@-t
- $(AM_V_at)chmod a+rx $@-t && mv -f $@-t $@
-
-MAINTAINERCLEANFILES += $(parallel_tests)
-EXTRA_DIST += gen-parallel-tests
-
-
-include $(srcdir)/instspc-tests.am
-
-$(srcdir)/instspc-tests.am: instspc-tests.sh Makefile.am
- $(AM_V_GEN)($(am__cd) $(srcdir) \
- && $(SHELL) ./instspc-tests.sh --generate-makefile) >$@
-
-$(instspc_tests): Makefile.am
- $(AM_V_at)rm -f $@ $@-t
- $(AM_V_GEN) :; \
- base=`expr 'x/$@' : 'x.*/instspc-\(.*\)\.test$$'`; \
- name=`expr x"$$base" : x'\(.*\)-'`; \
- action=`expr x"$$base" : x'.*-\(.*\)'`; \
- { \
- echo '#!/bin/sh'; \
- echo '# DO NOT EDIT! GENERATED AUTOMATICALLY!'; \
- echo; \
- echo '# Ensure proper definition of $$testsrcdir.'; \
- echo '. ./defs-static || exit 99'; \
- echo 'test -n "$$testsrcdir" || exit 99 # sanity check'; \
- echo; \
- echo "instspc_test_name='$$name'"; \
- echo "instspc_action='test-$$action'"; \
- echo ". \$$testsrcdir/instspc-tests.sh"; \
- } > $@-t
- $(AM_V_at)chmod a+rx $@-t && mv -f $@-t $@
-
-# All instspc-*.test tests work by sourcing the `instspc-tests.sh'
-# script. Also, they all use shared data generated by the helper
-# test `instspc-data.test', for reasons of speed.
-instspc-data.log: instspc-tests.sh
-$(instspc_tests:.test=.log): instspc-tests.sh instspc-data.log
-
-MAINTAINERCLEANFILES += $(instspc_tests)
-EXTRA_DIST += instspc-tests.sh
-XFAIL_TESTS += $(instspc_xfail_tests)
-
-
# Some testsuite-influential variables should be overridable from the
# test scripts, but not from the environment.
AM_TESTS_ENVIRONMENT = \
test x"$$me" = x || unset me; \
test x"$$required" = x || unset required; \
+ test x"$$using_tap" = x || unset using_tap; \
test x"$$parallel_tests" = x || unset parallel_tests; \
+ test x"$$test_prefer_config_shell" || unset test_prefer_config_shell; \
test x"$$original_AUTOMAKE" = x || unset original_AUTOMAKE; \
test x"$$original_ACLOCAL" = x || unset original_ACLOCAL;
+# The `AM_TESTS_REEXEC=no' setting tells the tests not to needlessly
+# re-execute themselves with the shell detected at configure time, since
+# we are already running them under it explicitly in our setup (see e.g.
+# the definition of TEST_LOG_COMPILER above).
+AM_TESTS_ENVIRONMENT += AM_TESTS_REEXEC=no; export AM_TESTS_REEXEC;
+# We want warning messages and explanations for skipped tests to go to
+# the console if possible, so set up `stderr_fileno_' properly.
+AM_TESTS_FD_REDIRECT = 9>&2
+AM_TESTS_ENVIRONMENT += stderr_fileno_=9; export stderr_fileno_;
-TESTS = \
-aclocal.test \
-aclocal3.test \
-aclocal4.test \
-aclocal5.test \
-aclocal6.test \
-aclocal7.test \
-aclocal8.test \
-aclocal9.test \
-acloca10.test \
-acloca11.test \
-acloca12.test \
-acloca13.test \
-acloca14.test \
-acloca15.test \
-acloca16.test \
-acloca17.test \
-acloca18.test \
-acloca19.test \
-acloca20.test \
-acloca21.test \
-acloca22.test \
-acloca23.test \
-acoutnoq.test \
-acoutpt.test \
-acoutpt2.test \
-acoutqnl.test \
-acoutbs.test \
-acoutbs2.test \
-acsilent.test \
-acsubst.test \
-acsubst2.test \
-all.test \
-all2.test \
-alloca.test \
-alloca2.test \
-alpha.test \
-alpha2.test \
-amhello-cflags.test \
-amhello-cross-compile.test \
-amhello-binpkg.test \
-amassign.test \
-ammissing.test \
-amopt.test \
-amopts-location.test \
-amopts-variable-expansion.test \
-amsubst.test \
-ansi2knr-no-more.test \
-ar-lib.test \
-ar.test \
-ar2.test \
-asm.test \
-asm2.test \
-asm3.test \
-autodist.test \
-autodist-subdir.test \
-autodist-acconfig.test \
-autodist-acconfig-no-subdir.test \
-autodist-aclocal-m4.test \
-autodist-config-headers.test \
-autodist-configure-no-subdir.test \
-autodist-no-duplicate.test \
-autodist-stamp-vti.test \
-autohdr.test \
-autohdr2.test \
-autohdr3.test \
-autohdr4.test \
-autohdrdry.test \
-automake.test \
-auxdir.test \
-auxdir2.test \
-auxdir3.test \
-auxdir4.test \
-auxdir5.test \
-auxdir6.test \
-auxdir7.test \
-auxdir8.test \
-auxdir9.test \
-backcompat.test \
-backcompat2.test \
-backcompat3.test \
-backcompat4.test \
-backcompat5.test \
-backcompat6.test \
-backsl.test \
-backsl2.test \
-backsl3.test \
-backsl4.test \
-badline.test \
-badopt.test \
-badprog.test \
-block.test \
-bsource.test \
-candist.test \
-canon.test \
-canon2.test \
-canon3.test \
-canon4.test \
-canon5.test \
-canon6.test \
-canon7.test \
-canon8.test \
-canon-name.test \
-ccnoco.test \
-ccnoco2.test \
-ccnoco3.test \
-check.test \
-check2.test \
-check3.test \
-check4.test \
-check5.test \
-check6.test \
-check7.test \
-check8.test \
-check9.test \
-check10.test \
-check11.test \
-check12.test \
-check-exported-srcdir.test \
-check-tests-in-builddir.test \
-check-tests_environment.test \
-check-concurrency-bug9245.test \
-tests-environment-backcompat.test \
-checkall.test \
-clean.test \
-clean2.test \
-colneq.test \
-colneq2.test \
-colneq3.test \
-colon.test \
-colon2.test \
-colon3.test \
-colon4.test \
-colon5.test \
-colon6.test \
-colon7.test \
-color.test \
-color2.test \
-comment.test \
-comment2.test \
-comment3.test \
-comment4.test \
-comment5.test \
-comment6.test \
-comment7.test \
-comment8.test \
-comment9.test \
-commen10.test \
-commen11.test \
-comments-in-var-def.test \
-compile.test \
-compile2.test \
-compile3.test \
-compile4.test \
-compile5.test \
-compile6.test \
-compile_f90_c_cxx.test \
-compile_f_c_cxx.test \
-cond-basic.test \
-cond.test \
-cond2.test \
-cond3.test \
-cond4.test \
-cond5.test \
-cond6.test \
-cond7.test \
-cond8.test \
-cond9.test \
-cond10.test \
-cond11.test \
-cond13.test \
-cond14.test \
-cond15.test \
-cond16.test \
-cond17.test \
-cond18.test \
-cond19.test \
-cond20.test \
-cond21.test \
-cond22.test \
-cond23.test \
-cond24.test \
-cond25.test \
-cond26.test \
-cond27.test \
-cond28.test \
-cond29.test \
-cond30.test \
-cond31.test \
-cond32.test \
-cond33.test \
-cond34.test \
-cond35.test \
-cond36.test \
-cond37.test \
-cond38.test \
-cond39.test \
-cond40.test \
-cond41.test \
-cond42.test \
-cond43.test \
-cond44.test \
-cond45.test \
-cond46.test \
-condd.test \
-condhook.test \
-condhook2.test \
-condinc.test \
-condinc2.test \
-condlib.test \
-condman.test \
-condman2.test \
-condman3.test \
-configure.test \
-confdeps.test \
-conff.test \
-conff2.test \
-confh.test \
-confh4.test \
-confh5.test \
-confh6.test \
-confh7.test \
-confh8.test \
-confincl.test \
-conflnk.test \
-conflnk2.test \
-conflnk3.test \
-conflnk4.test \
-confsub.test \
-confvar.test \
-confvar2.test \
-copy.test \
-cscope.test \
-cscope2.test \
-cscope3.test \
-cxx.test \
-cxx2.test \
-cxxcpp.test \
-cxxlibobj.test \
-cxxlink.test \
-cxxnoc.test \
-cxxo.test \
-cygnus-check-without-all.test \
-cygnus-dependency-tracking.test \
-cygnus-imply-foreign.test \
-cygnus-no-dist.test \
-cygnus-no-installinfo.test \
-cygnus-requires-maintainer-mode.test \
-cygwin32.test \
-dash.test \
-defun.test \
-defun2.test \
-dejagnu.test \
-dejagnu2.test \
-dejagnu3.test \
-dejagnu4.test \
-dejagnu5.test \
-dejagnu6.test \
-dejagnu7.test \
-depacl2.test \
-depcomp.test \
-depcomp2.test \
-depcomp3.test \
-depcomp4.test \
-depcomp5.test \
-depcomp6.test \
-depcomp7.test \
-depcomp8a.test \
-depcomp8b.test \
-depcomp9.test \
-depcomp10.test \
-depdist.test \
-depend.test \
-depend2.test \
-depend3.test \
-depend4.test \
-depend5.test \
-depend6.test \
-deprecated-acinit.test \
-destdir.test \
-dirlist.test \
-dirlist2.test \
-discover.test \
-dist-included-parent-dir.test \
-distcleancheck.test \
-distcom2.test \
-distcom3.test \
-distcom4.test \
-distcom5.test \
-distcom6.test \
-distcom7.test \
-distdir.test \
-distlinks.test \
-distlinksbrk.test \
-distname.test \
-distcheck-configure-flags.test \
-distcheck-configure-flags-am.test \
-distcheck-configure-flags-subpkg.test \
-distcheck-hook.test \
-distcheck-hook2.test \
-dmalloc.test \
-dollar.test \
-dollarvar.test \
-dollarvar2.test \
-double.test \
-dup2.test \
-else.test \
-empty.test \
-empty2.test \
-empty3.test \
-empty4.test \
-exdir.test \
-exdir2.test \
-exdir3.test \
-exeext.test \
-exeext2.test \
-exeext3.test \
-exeext4.test \
-exsource.test \
-ext.test \
-ext2.test \
-ext3.test \
-extra.test \
-extra2.test \
-extra3.test \
-extra4.test \
-extra5.test \
-extra6.test \
-extra7.test \
-extra8.test \
-extra9.test \
-extra10.test \
-extra11.test \
-extra12.test \
-extradep.test \
-extradep2.test \
-f90only.test \
-flavor.test \
-flibs.test \
-fn99.test \
-fn99subdir.test \
-fnoc.test \
-fo.test \
-forcemiss.test \
-forcemiss2.test \
-fort1.test \
-fort2.test \
-fort4.test \
-fort5.test \
-fonly.test \
-fortdep.test \
-gcj.test \
-gcj2.test \
-gcj3.test \
-gcj4.test \
-gcj5.test \
-gcj6.test \
-getopt.test \
-gettext.test \
-gettext2.test \
-gettext3.test \
-gnumake.test \
-gnuwarn.test \
-gnuwarn2.test \
-gnits.test \
-gnits2.test \
-gnits3.test \
-hdr-vars-defined-once.test \
-header.test \
-help.test \
-help2.test \
-help3.test \
-help4.test \
-help-depend.test \
-help-depend2.test \
-help-dmalloc.test \
-help-init.test \
-help-lispdir.test \
-help-maintainer.test \
-help-multilib.test \
-help-python.test \
-help-regex.test \
-help-silent.test \
-help-upc.test \
-hfs.test \
-hosts.test \
-implicit.test \
-info.test \
-init.test \
-init2.test \
-insh2.test \
-install2.test \
-installdir.test \
-instsh.test \
-instsh2.test \
-instsh3.test \
-instdat.test \
-instdat2.test \
-instdir.test \
-instdir2.test \
-instdir-java.test \
-instdir-lisp.test \
-instdir-ltlib.test \
-instdir-prog.test \
-instdir-python.test \
-instdir-texi.test \
-instexec.test \
-instfail.test \
-instfail-info.test \
-instfail-java.test \
-instfail-libtool.test \
-insthook.test \
-instman.test \
-instman2.test \
-instmany.test \
-instmany-mans.test \
-instmany-python.test \
-instspc-data.test \
-$(instspc_tests) \
-interp.test \
-interp2.test \
-java.test \
-java2.test \
-java3.test \
-java-check.test \
-java-empty-classpath.test \
-javaprim.test \
-javasubst.test \
-java-extra.test \
-java-noinst.test \
-ldadd.test \
-ldflags.test \
-lex.test \
-lex2.test \
-lex3.test \
-lex4.test \
-lex5.test \
-lexcpp.test \
-lexvpath.test \
-lex-subobj-nodep.test \
-lflags.test \
-lflags2.test \
-libexec.test \
-libobj-basic.test \
-libobj2.test \
-libobj3.test \
-libobj4.test \
-libobj5.test \
-libobj7.test \
-libobj10.test \
-libobj12.test \
-libobj13.test \
-libobj14.test \
-libobj15a.test \
-libobj15b.test \
-libobj15c.test \
-libobj16a.test \
-libobj16b.test \
-libobj17.test \
-libobj18.test \
-libobj19.test \
-libobj20a.test \
-libobj20b.test \
-libobj20c.test \
-library.test \
-library2.test \
-library3.test \
-libtool.test \
-libtool2.test \
-libtool3.test \
-libtool4.test \
-libtool5.test \
-libtool6.test \
-libtool7.test \
-libtool8.test \
-libtool9.test \
-libtoo10.test \
-libtoo11.test \
-license.test \
-license2.test \
-link_c_cxx.test \
-link_dist.test \
-link_f90_only.test \
-link_fc.test \
-link_fccxx.test \
-link_fcxx.test \
-link_f_only.test \
-link_override.test \
-lisp2.test \
-lisp3.test \
-lisp4.test \
-lisp5.test \
-lisp6.test \
-lisp7.test \
-lisp8.test \
-lispdry.test \
-listval.test \
-location.test \
-longline.test \
-longlin2.test \
-ltcond.test \
-ltcond2.test \
-ltconv.test \
-ltdeps.test \
-ltinit.test \
-ltinstloc.test \
-ltlibobjs.test \
-ltlibsrc.test \
-ltorder.test \
-lzip.test \
-lzma.test \
-m4-inclusion.test \
-maintclean.test \
-maintclean-vpath.test \
-make.test \
-makej.test \
-makej2.test \
-maken.test \
-maken2.test \
-maken3.test \
-maken4.test \
-makevars.test \
-man.test \
-man2.test \
-man3.test \
-man4.test \
-man5.test \
-man6.test \
-man7.test \
-man8.test \
-mdate.test \
-mdate2.test \
-mdate3.test \
-mdate4.test \
-mdate5.test \
-mdate6.test \
-missing.test \
-missing2.test \
-missing3.test \
-missing4.test \
-missing5.test \
-missing6.test \
-mkinstall.test \
-mkinst2.test \
-mkinst3.test \
-mmode.test \
-mmodely.test \
-multlib.test \
-no-extra-makefile-code.test \
-no-outdir-option.test \
-nobase.test \
-nobase-libtool.test \
-nobase-python.test \
-nodef.test \
-nodef2.test \
-nodep.test \
-nodep2.test \
-nodepcomp.test \
-nodist.test \
-nodist2.test \
-nodist3.test \
-nogzip.test \
-nogzip2.test \
-noinst.test \
-noinstdir.test \
-nolink.test \
-nostdinc.test \
-notrans.test \
-number.test \
-objc.test \
-objc2.test \
-obsolete.test \
-oldvars.test \
-order.test \
-output.test \
-output2.test \
-output3.test \
-output4.test \
-output5.test \
-output6.test \
-output7.test \
-output8.test \
-output9.test \
-output10.test \
-output11.test \
-output12.test \
-output13.test \
-output-order.test \
-override-conditional-1.test \
-override-conditional-2.test \
-override-html.test \
-override-suggest-local.test \
-parallel-am.test \
-parallel-am2.test \
-parallel-am3.test \
-parallel-tests.test \
-parallel-tests2.test \
-parallel-tests3.test \
-parallel-tests4.test \
-parallel-tests5.test \
-parallel-tests6.test \
-parallel-tests7.test \
-parallel-tests8.test \
-parallel-tests9.test \
-parallel-tests10.test \
-parallel-tests-am_tests_environment.test \
-parallel-tests-unreadable-log.test \
-parallel-tests-subdir.test \
-parallel-tests-interrupt.test \
-parallel-tests-reset-term.test \
-parallel-tests-harderror.test \
-parallel-tests-log-override-1.test \
-parallel-tests-log-override-2.test \
-parallel-tests-log-override-recheck.test \
-parallel-tests-cmdline-override.test \
-parallel-tests-log-compiler-example.test \
-parallel-tests-fork-bomb.test \
-parse.test \
-percent.test \
-percent2.test \
-phony.test \
-pluseq.test \
-pluseq2.test \
-pluseq3.test \
-pluseq4.test \
-pluseq5.test \
-pluseq6.test \
-pluseq7.test \
-pluseq8.test \
-pluseq9.test \
-pluseq10.test \
-pluseq11.test \
-posixsubst-data.test \
-posixsubst-extradist.test \
-posixsubst-ldadd.test \
-posixsubst-libraries.test \
-posixsubst-ltlibraries.test \
-posixsubst-programs.test \
-posixsubst-scripts.test \
-posixsubst-sources.test \
-posixsubst-tests.test \
-postproc.test \
-ppf77.test \
-pr2.test \
-pr9.test \
-pr72.test \
-pr87.test \
-pr204.test \
-pr211.test \
-pr220.test \
-pr224.test \
-pr229.test \
-pr243.test \
-pr266.test \
-pr279.test \
-pr279-2.test \
-pr287.test \
-pr300-lib.test \
-pr300-ltlib.test \
-pr300-prog.test \
-pr307.test \
-pr401.test \
-pr401b.test \
-pr401c.test \
-prefix.test \
-primary.test \
-primary2.test \
-primary3.test \
-primary-prefix-invalid-couples.test \
-primary-prefix-valid-couples.test \
-primary-prefix-couples-force-valid.test \
-primary-prefix-couples-documented-valid.test \
-proginst.test \
-python.test \
-python2.test \
-python3.test \
-python4.test \
-python5.test \
-python5b.test \
-python6.test \
-python7.test \
-python8.test \
-python9.test \
-python10.test \
-python11.test \
-python12.test \
-python-dist.test \
-python-vars.test \
-python-virtualenv.test \
-recurs.test \
-recurs2.test \
-remake.test \
-remake1a.test \
-remake2.test \
-remake3.test \
-remake3a.test \
-remake4.test \
-remake5.test \
-remake6.test \
-remake7.test \
-remake8a.test \
-remake8b.test \
-remake9a.test \
-remake9b.test \
-remake9c.test \
-remake9d.test \
-remake10a.test \
-remake10b.test \
-remake10c.test \
-remake11.test \
-remake12.test \
-remake-subdir-from-subdir.test \
-remake-subdir-gnu.test \
-remake-subdir.test \
-remake-subdir2.test \
-remake-subdir-long-time.test \
-remake-gnulib-add-acsubst.test \
-remake-gnulib-add-header.test \
-remake-gnulib-remove-header.test \
-remake-moved-m4-file.test \
-remake-deleted-m4-file.test \
-remake-renamed-m4-file.test \
-remake-renamed-m4-macro-and-file.test \
-remake-renamed-m4-macro.test \
-pr8365-remake-timing.test \
-regex.test \
-regex-obsolete.test \
-req.test \
-reqd.test \
-reqd2.test \
-repeated-options.test \
-rulepat.test \
-self-check-cleanup.test \
-self-check-dir.test \
-self-check-env-sanitize.test \
-self-check-exit.test \
-self-check-is_newest.test \
-self-check-me.test \
-self-check-report.test \
-self-check-sanity.test \
-self-check-unindent.test \
-sanity.test \
-scripts.test \
-seenc.test \
-silent.test \
-silent2.test \
-silent3.test \
-silent4.test \
-silent6.test \
-silent7.test \
-silent8.test \
-silent9.test \
-silentcxx.test \
-silentf77.test \
-silentf90.test \
-silent-many-gcc.test \
-silent-many-generic.test \
-silent-lex-gcc.test \
-silent-lex-generic.test \
-silent-yacc-gcc.test \
-silent-yacc-generic.test \
-silent-nowarn.test \
-silent-configsite.test \
-srcsub.test \
-srcsub2.test \
-space.test \
-specflg.test \
-specflg2.test \
-specflg3.test \
-specflg6.test \
-specflg7.test \
-specflg8.test \
-specflg9.test \
-specflg10.test \
-specflg-dummy.test \
-spell.test \
-spell2.test \
-spell3.test \
-spelling.test \
-spy.test \
-stdinc.test \
-stamph2.test \
-stdlib.test \
-stdlib2.test \
-strictness-override.test \
-strictness-precedence.test \
-strip.test \
-strip2.test \
-strip3.test \
-subdir.test \
-subdir2.test \
-subdir3.test \
-subdir4.test \
-subdir5.test \
-subdir6.test \
-subdir7.test \
-subdir8.test \
-subdir9.test \
-subdir10.test \
-subdirbuiltsources.test \
-subcond.test \
-subcond2.test \
-subcond3.test \
-subobj.test \
-subobj2.test \
-subobj4.test \
-subobj5.test \
-subobj6.test \
-subobj7.test \
-subobj8.test \
-subobj9.test \
-subobj10.test \
-subobj11a.test \
-subobj11b.test \
-subobj11c.test \
-subobjname.test \
-subpkg.test \
-subpkg2.test \
-subpkg3.test \
-subpkg4.test \
-subst.test \
-subst2.test \
-subst3.test \
-subst4.test \
-subst5.test \
-substref.test \
-substre2.test \
-substtarg.test \
-suffix.test \
-suffix2.test \
-suffix3.test \
-suffix4.test \
-suffix5.test \
-suffix6.test \
-suffix6b.test \
-suffix6c.test \
-suffix7.test \
-suffix8.test \
-suffix9.test \
-suffix10.test \
-suffix11.test \
-suffix12.test \
-suffix13.test \
-suffix-chain.test \
-symlink.test \
-symlink2.test \
-syntax.test \
-tags.test \
-tags2.test \
-tagsub.test \
-tar.test \
-tar2.test \
-tar3.test \
-target-cflags.test \
-targetclash.test \
-tests-environment-fd-redirect.test \
-tests-environment-and-log-compiler.test \
-txinfo.test \
-txinfo2.test \
-txinfo3.test \
-txinfo4.test \
-txinfo5.test \
-txinfo5b.test \
-txinfo6.test \
-txinfo7.test \
-txinfo8.test \
-txinfo9.test \
-txinfo10.test \
-txinfo13.test \
-txinfo16.test \
-txinfo17.test \
-txinfo18.test \
-txinfo19.test \
-txinfo20.test \
-txinfo21.test \
-txinfo22.test \
-txinfo23.test \
-txinfo24.test \
-txinfo25.test \
-txinfo26.test \
-txinfo27.test \
-txinfo28.test \
-txinfo29.test \
-txinfo30.test \
-txinfo31.test \
-txinfo32.test \
-txinfo33.test \
-txinfo-unrecognized-extension.test \
-transform.test \
-transform2.test \
-unused.test \
-upc.test \
-upc2.test \
-upc3.test \
-vala.test \
-vala1.test \
-vala2.test \
-vala3.test \
-vala4.test \
-vala5.test \
-vars.test \
-vars3.test \
-vartar.test \
-vartypos.test \
-vartypo2.test \
-version.test \
-version2.test \
-version3.test \
-version4.test \
-version6.test \
-version7.test \
-version8.test \
-vpath.test \
-vtexi.test \
-vtexi2.test \
-vtexi3.test \
-vtexi4.test \
-warnings-override.test \
-warnings-precedence.test \
-warnings-strictness-interactions.test \
-warnings-unknown.test \
-warnopts.test \
-warnings-win-over-strictness.test \
-warning-groups-win-over-strictness.test \
-werror.test \
-werror2.test \
-werror3.test \
-werror4.test \
-whoami.test \
-xsource.test \
-xz.test \
-yacc-basic.test \
-yacc-d-basic.test \
-yacc-clean.test \
-yacc.test \
-yacc2.test \
-yacc4.test \
-yacc5.test \
-yacc6.test \
-yacc7.test \
-yacc8.test \
-yaccdry.test \
-yacc-dist-nobuild.test \
-yacc-nodist.test \
-yaccpp.test \
-yaccvpath.test \
-yacc-d-vpath.test \
-yacc-dist-nobuild-subdir.test \
-yflags.test \
-yflags2.test \
-yflags-cmdline-override.test \
-yflags-conditional.test \
-yflags-d-false-positives.test \
-yflags-force-override.test \
-yflags-force-conditional.test \
-yflags-var-expand.test \
-$(parallel_tests)
+# Hand-written tests.
-EXTRA_DIST += $(TESTS)
+include $(srcdir)/list-of-tests.mk
+TESTS += $(handwritten_TESTS)
+EXTRA_DIST += $(handwritten_TESTS)
+
+# "Auto-generated" tests wrapping hand-written ones.
+
+include $(srcdir)/wrap-tests.am
+
+TESTS += $(wrap_TESTS)
+
+$(srcdir)/wrap-tests.am: gen-wrap-tests list-of-tests.mk Makefile.am
+ $(AM_V_GEN)($(am__cd) $(srcdir) \
+ && $(SHELL) ./gen-wrap-tests) > wrap-tests.tmp
+ $(AM_V_at)chmod a-w wrap-tests.tmp && mv -f wrap-tests.tmp $@
+EXTRA_DIST += gen-wrap-tests
+
+# These tests are not expected to exist on filesystem, so we need this
+# declaration to avoid spurious failures.
+$(wrap_TESTS):
+# All the "wrapper" tests work by sourcing the script `wrap-tests.sh'.
+$(wrap_LOGS): wrap-tests.sh
+EXTRA_DIST += wrap-tests.sh
+
+# Extra dependencies for hand-written tests.
+
+test-driver-custom-multitest.log: trivial-test-driver
+test-driver-custom-multitest-recheck.log: trivial-test-driver
+test-driver-custom-multitest-recheck2.log: trivial-test-driver
+test-driver-custom-html.log: trivial-test-driver
+EXTRA_DIST += trivial-test-driver
+
+testsuite-summary-color.log testsuite-summary-count.log: \
+ testsuite-summary-checks.sh extract-testsuite-summary
+EXTRA_DIST += testsuite-summary-checks.sh
+EXTRA_DIST += extract-testsuite-summary
+
+testsuite-summary-count-many.log: trivial-test-driver
+testsuite-summary-count-many.log: extract-testsuite-summary
+
+$(tap_with_common_setup_TESTS:.test=.log): tap-common-setup.log tap-setup.sh
+EXTRA_DIST += tap-setup.sh
+
+tap-summary.log tap-summary-color.log: tap-summary-aux.sh
+EXTRA_DIST += tap-summary-aux.sh
+
+# If two test scripts have the same basename, they will end up sharing
+# the same log file, leading to all sort of undefined and undesired
+# behaviours.
+check-no-repeated-test-name:
+ @LC_ALL=C; export LC_ALL; \
+ lst='$(TEST_LOGS)'; for log in $$lst; do echo $$log; done \
+ | sort | uniq -c | awk '($$1 > 1) { print }' \
+ | sed 's/\.log$$//' | grep . >&2 \
+ && { \
+ echo $@: test names listed above are duplicated >&2; \
+ exit 1; \
+ }; :
+check-local: check-no-repeated-test-name
+.PHONY: check-no-repeated-test-name
+
+## Checking the list of tests.
+checked_test_extensions = .test .tap
+expected_list_of_tests = $(handwritten_TESTS)
+include $(top_srcdir)/CheckListOfTests.am
# Dependencies valid for each test case.
$(TEST_LOGS): defs defs-static aclocal-$(APIVERSION) automake-$(APIVERSION)
+# FIXME: this should be made more granular once we have a cleaner
+# subdivision of the tests.
+$(TEST_LOGS): plain-functions.sh tap-functions.sh
+
+## Files containing auxiliary functions used by our test cases.
+EXTRA_DIST += tap-functions.sh plain-functions.sh
clean-local: clean-local-check
.PHONY: clean-local-check
PRE_UNINSTALL = :
POST_UNINSTALL = :
build_triplet = @build@
+host_triplet = @host@
DIST_COMMON = README $(srcdir)/Makefile.am $(srcdir)/Makefile.in \
$(srcdir)/aclocal.in $(srcdir)/automake.in $(srcdir)/defs \
- $(srcdir)/defs-static.in $(srcdir)/instspc-tests.am \
- $(srcdir)/parallel-tests.am
+ $(srcdir)/defs-static.in $(srcdir)/list-of-tests.mk \
+ $(srcdir)/wrap-tests.am $(top_srcdir)/CheckListOfTests.am
subdir = tests
ACLOCAL_M4 = $(top_srcdir)/aclocal.m4
am__aclocal_m4_deps = $(top_srcdir)/m4/amversion.m4 \
am__v_at_0 = @
SOURCES =
DIST_SOURCES =
+am__tty_colors_dummy = \
+ mgn= red= grn= lgn= blu= brg= std=; \
+ am__color_tests=no
# If stdout is a non-dumb tty, use colors. If test -t is not supported,
# then this fails; a conservative approach. Of course do not redirect
# stdout here, just stderr.
am__tty_colors = \
-red=; grn=; lgn=; blu=; std=; \
+$(am__tty_colors_dummy); \
test "X$(AM_COLOR_TESTS)" != Xno \
&& test "X$$TERM" != Xdumb \
&& { test "X$(AM_COLOR_TESTS)" = Xalways || test -t 1 2>/dev/null; } \
&& { \
+ am__color_tests=yes; \
red='\e[0;31m'; \
grn='\e[0;32m'; \
lgn='\e[1;32m'; \
blu='\e[1;34m'; \
+ mgn='\e[0;35m'; \
+ brg='\e[1m'; \
std='\e[m'; \
}
am__vpath_adj_setup = srcdirstrip=`echo "$(srcdir)" | sed 's|.|.|g'`;
# Restructured Text title and section.
am__rst_title = sed 's/.*/ & /;h;s/./=/g;p;x;p;g;p;s/.*//'
am__rst_section = sed 'p;s/./=/g;p;g'
-# Put stdin (possibly several lines separated by ". ") in a box.
-# Prefix each line by 'col' and terminate each with 'std', for coloring.
-# Multi line coloring is problematic with "less -R", so we really need
-# to color each line individually.
-am__text_box = $(AWK) '{ \
- n = split($$0, lines, "\\. "); max = 0; \
- for (i = 1; i <= n; ++i) \
- if (max < length(lines[i])) \
- max = length(lines[i]); \
- for (i = 0; i < max; ++i) \
- line = line "="; \
- print col line std; \
- for (i = 1; i <= n; ++i) \
- if (lines[i]) \
- print col lines[i] std; \
- print col line std; \
-}'
# Solaris 10 'make', and several other traditional 'make' implementations,
# pass "-e" to $(SHELL), and POSIX 2008 even requires this. Work around it
# by disabling -e (using the XSI extension "set +e") if it's set.
am__sh_e_setup = case $$- in *e*) set +e;; esac
+# Default flags passed to test drivers.
+am__common_driver_flags = \
+ --color-tests "$$am__color_tests" \
+ --enable-hard-errors "$$am__enable_hard_errors" \
+ --expect-failure "$$am__expect_failure"
# To be inserted before the command running the test. Creates the
# directory for the log if needed. Stores in $dir the directory
# containing $f, in $tst the test, in $log the log. Executes the
# developer- defined test setup AM_TESTS_ENVIRONMENT (if any), and
-# passes TESTS_ENVIRONMENT. Saves and restores TERM around uses of
-# TESTS_ENVIRONMENT and AM_TESTS_ENVIRONMENT, in case any of them
-# unsets it.
+# passes TESTS_ENVIRONMENT. Set up options for the wrapper that
+# will run the test scripts (or their associated LOG_COMPILER, if
+# thy have one).
am__check_pre = \
$(am__sh_e_setup); \
$(am__vpath_adj_setup) $(am__vpath_adj) \
+$(am__tty_colors); \
srcdir=$(srcdir); export srcdir; \
-rm -f $@-t; \
-am__trap='rm -f '\''$(abs_builddir)/$@-t'\''; (exit $$st); exit $$st'; \
-trap "st=129; $$am__trap" 1; trap "st=130; $$am__trap" 2; \
-trap "st=141; $$am__trap" 13; trap "st=143; $$am__trap" 15; \
am__odir=`echo "./$@" | sed 's|/[^/]*$$||'`; \
test "x$$am__odir" = x. || $(MKDIR_P) "$$am__odir" || exit $$?; \
if test -f "./$$f"; then dir=./; \
elif test -f "$$f"; then dir=; \
else dir="$(srcdir)/"; fi; \
-tst=$$dir$$f; log='$@'; __SAVED_TERM=$$TERM; \
-$(AM_TESTS_ENVIRONMENT) \
-$(TESTS_ENVIRONMENT)
-# To be appended to the command running the test. Handle the stdout
-# and stderr redirection, and catch the exit status.
-am__check_post = \
->$@-t 2>&1; \
-estatus=$$?; \
-if test -n '$(DISABLE_HARD_ERRORS)' \
- && test $$estatus -eq 99; then \
- estatus=1; \
-fi; \
-TERM=$$__SAVED_TERM; export TERM; \
-$(am__tty_colors); \
-xfailed=PASS; \
+tst=$$dir$$f; log='$@'; \
+if test -n '$(DISABLE_HARD_ERRORS)'; then \
+ am__enable_hard_errors=no; \
+else \
+ am__enable_hard_errors=yes; \
+fi; \
case " $(XFAIL_TESTS) " in \
*[\ \ ]$$f[\ \ ]* | *[\ \ ]$$dir$$f[\ \ ]*) \
- xfailed=XFAIL;; \
-esac; \
-case $$estatus.$$xfailed in \
- 0.XFAIL) col=$$red; res=XPASS;; \
- 0.*) col=$$grn; res=PASS ;; \
- 77.*) col=$$blu; res=SKIP ;; \
- 99.*) col=$$red; res=FAIL ;; \
- *.XFAIL) col=$$lgn; res=XFAIL;; \
- *.*) col=$$red; res=FAIL ;; \
-esac; \
-echo "$${col}$$res$${std}: $$f"; \
-echo "$$res: $$f (exit: $$estatus)" | \
- $(am__rst_section) >$@; \
-cat $@-t >>$@; \
-rm -f $@-t
+ am__expect_failure=yes;; \
+ *) \
+ am__expect_failure=no;; \
+esac; \
+$(AM_TESTS_ENVIRONMENT) $(TESTS_ENVIRONMENT)
+# A shell command to get the names of the tests scripts with any registered
+# extension removed (i.e., equivalently, the names of the test logs, with
+# the `.log' extension removed). The result is saved in the shell variable
+# `$bases'. This honors runtime overriding of TESTS and TEST_LOGS. Sadly,
+# we cannot use something simpler, involving e.g., "$(TEST_LOGS:.log=)",
+# since that might cause problem with VPATH rewrites for suffix-less tests.
+# See also 'test-harness-vpath-rewrite.test' and 'test-trs-basic.test'.
+am__set_TESTS_bases = \
+ bases='$(TEST_LOGS)'; \
+ bases=`for i in $$bases; do echo $$i; done | sed 's/\.log$$//'`; \
+ bases=`echo $$bases`
RECHECK_LOGS = $(TEST_LOGS)
AM_RECURSIVE_TARGETS = check check-html recheck recheck-html
TEST_SUITE_LOG = test-suite.log
TEST_SUITE_HTML = $(TEST_SUITE_LOG:.log=.html)
-TEST_EXTENSIONS = .test
am__test_logs1 = $(TESTS:=.log)
-TEST_LOGS = $(am__test_logs1:.test.log=.log)
+am__test_logs2 = $(am__test_logs1:.test.log=.log)
+TEST_LOG_DRIVER = $(SHELL) $(top_srcdir)/lib/test-driver
TEST_LOG_COMPILE = $(TEST_LOG_COMPILER) $(AM_TEST_LOG_FLAGS) \
$(TEST_LOG_FLAGS)
-TEST_LOGS_TMP = $(TEST_LOGS:.log=.log-t)
+am__test_logs3 = $(am__test_logs2:.tap.log=.log)
+TAP_LOG_COMPILE = $(TAP_LOG_COMPILER) $(AM_TAP_LOG_FLAGS) \
+ $(TAP_LOG_FLAGS)
+am__test_logs4 = $(am__test_logs3:.test_pt.log=.log)
+TEST_PT_LOG_COMPILE = $(TEST_PT_LOG_COMPILER) $(AM_TEST_PT_LOG_FLAGS) \
+ $(TEST_PT_LOG_FLAGS)
+am__test_logs5 = $(am__test_logs4:.test_cs.log=.log)
+TEST_CS_LOG_COMPILE = $(TEST_CS_LOG_COMPILER) $(AM_TEST_CS_LOG_FLAGS) \
+ $(TEST_CS_LOG_FLAGS)
+am__test_logs6 = $(am__test_logs5:.test_pltap.log=.log)
+TEST_PLTAP_LOG_COMPILE = $(TEST_PLTAP_LOG_COMPILER) \
+ $(AM_TEST_PLTAP_LOG_FLAGS) $(TEST_PLTAP_LOG_FLAGS)
+TEST_LOGS = $(am__test_logs6:.tap_pltap.log=.log)
+TAP_PLTAP_LOG_COMPILE = $(TAP_PLTAP_LOG_COMPILER) \
+ $(AM_TAP_PLTAP_LOG_FLAGS) $(TAP_PLTAP_LOG_FLAGS)
DISTFILES = $(DIST_COMMON) $(DIST_SOURCES) $(TEXINFOS) $(EXTRA_DIST)
ACLOCAL = @ACLOCAL@
AMTAR = @AMTAR@
docdir = @docdir@
dvidir = @dvidir@
exec_prefix = @exec_prefix@
+host = @host@
host_alias = @host_alias@
+host_cpu = @host_cpu@
+host_os = @host_os@
+host_vendor = @host_vendor@
htmldir = @htmldir@
includedir = @includedir@
infodir = @infodir@
top_build_prefix = @top_build_prefix@
top_builddir = @top_builddir@
top_srcdir = @top_srcdir@
-MAINTAINERCLEANFILES = $(parallel_tests) $(instspc_tests)
-EXTRA_DIST = ChangeLog-old gen-parallel-tests instspc-tests.sh \
- $(TESTS)
-XFAIL_TESTS = all.test auxdir2.test cond17.test gcj6.test \
- override-conditional-2.test pr8365-remake-timing.test \
- yacc-dist-nobuild-subdir.test txinfo5.test \
- $(instspc_xfail_tests)
-parallel_tests = backcompat5-p.test check-concurrency-bug9245-p.test \
- check-exported-srcdir-p.test check-tests-in-builddir-p.test \
- check-tests_environment-p.test check-p.test check10-p.test \
- check11-p.test check12-p.test check2-p.test check3-p.test \
- check4-p.test check5-p.test check6-p.test check7-p.test \
- check8-p.test check9-p.test color-p.test color2-p.test \
- comment9-p.test dejagnu-p.test exeext4-p.test maken3-p.test \
- maken4-p.test posixsubst-tests-p.test
-instspc_tests = instspc-squote-build.test instspc-squote-install.test \
- instspc-dquote-build.test instspc-dquote-install.test \
- instspc-bquote-build.test instspc-bquote-install.test \
- instspc-sharp-build.test instspc-sharp-install.test \
- instspc-dollar-build.test instspc-dollar-install.test \
- instspc-bang-build.test instspc-bang-install.test \
- instspc-bslash-build.test instspc-bslash-install.test \
- instspc-ampersand-build.test instspc-ampersand-install.test \
- instspc-percent-build.test instspc-percent-install.test \
- instspc-leftpar-build.test instspc-leftpar-install.test \
- instspc-rightpar-build.test instspc-rightpar-install.test \
- instspc-pipe-build.test instspc-pipe-install.test \
- instspc-caret-build.test instspc-caret-install.test \
- instspc-tilde-build.test instspc-tilde-install.test \
- instspc-qmark-build.test instspc-qmark-install.test \
- instspc-star-build.test instspc-star-install.test \
- instspc-plus-build.test instspc-plus-install.test \
- instspc-minus-build.test instspc-minus-install.test \
- instspc-comma-build.test instspc-comma-install.test \
- instspc-colon-build.test instspc-colon-install.test \
- instspc-semicol-build.test instspc-semicol-install.test \
- instspc-equal-build.test instspc-equal-install.test \
- instspc-less-build.test instspc-less-install.test \
- instspc-more-build.test instspc-more-install.test \
- instspc-at-build.test instspc-at-install.test \
- instspc-lqbrack-build.test instspc-lqbrack-install.test \
- instspc-rqbrack-build.test instspc-rqbrack-install.test \
- instspc-lcbrack-build.test instspc-lcbrack-install.test \
- instspc-rcbrack-build.test instspc-rcbrack-install.test \
- instspc-space-build.test instspc-space-install.test \
- instspc-tab-build.test instspc-tab-install.test \
- instspc-linefeed-build.test instspc-linefeed-install.test \
- instspc-backspace-build.test instspc-backspace-install.test \
- instspc-formfeed-build.test instspc-formfeed-install.test \
- instspc-carriageret-build.test \
- instspc-carriageret-install.test \
- instspc-quadrigraph0-build.test \
- instspc-quadrigraph0-install.test \
- instspc-quadrigraph1-build.test \
- instspc-quadrigraph1-install.test \
- instspc-quadrigraph2-build.test \
- instspc-quadrigraph2-install.test \
- instspc-quadrigraph3-build.test \
- instspc-quadrigraph3-install.test \
- instspc-quadrigraph4-build.test \
- instspc-quadrigraph4-install.test instspc-a_b-build.test \
- instspc-a_b-install.test instspc-a__b-build.test \
- instspc-a__b-install.test instspc-a_lf_b-build.test \
- instspc-a_lf_b-install.test instspc-dotdotdot-build.test \
- instspc-dotdotdot-install.test instspc-dosdrive-build.test \
- instspc-dosdrive-install.test instspc-miscglob1-build.test \
- instspc-miscglob1-install.test instspc-miscglob2-build.test \
- instspc-miscglob2-install.test
-instspc_xfail_tests = instspc-squote-build.test \
- instspc-dquote-build.test instspc-bquote-build.test \
- instspc-sharp-build.test instspc-dollar-build.test \
- instspc-bslash-build.test instspc-ampersand-build.test \
- instspc-linefeed-build.test instspc-quadrigraph0-build.test \
- instspc-a_lf_b-build.test instspc-squote-install.test \
- instspc-dquote-install.test instspc-bquote-install.test \
- instspc-sharp-install.test instspc-dollar-install.test \
- instspc-linefeed-install.test instspc-a_lf_b-install.test
+
+# Run the tests with the shell detected at configure time.
+LOG_COMPILER = $(SHELL)
+TEST_EXTENSIONS = .test .tap .test_pt .test_cs .test_pltap .tap_pltap
+TEST_LOG_COMPILER = $(LOG_COMPILER)
+TAP_LOG_COMPILER = $(LOG_COMPILER)
+TAP_LOG_DRIVER = AM_TAP_AWK='$(AWK)' $(SHELL) $(top_srcdir)/lib/tap-driver.sh
+AM_TAP_LOG_DRIVER_FLAGS = --merge --comments --diagnostic-string \
+ `printf '\043%%\043\n'`
+EXTRA_DIST = ChangeLog-old $(handwritten_TESTS) gen-wrap-tests \
+ wrap-tests.sh trivial-test-driver testsuite-summary-checks.sh \
+ extract-testsuite-summary tap-setup.sh tap-summary-aux.sh \
+ tap-functions.sh plain-functions.sh
+
+# Hand-written tests.
+
+# "Auto-generated" tests wrapping hand-written ones.
+TESTS = $(handwritten_TESTS) $(wrap_TESTS)
+XFAIL_TESTS = \
+all.test \
+cond17.test \
+gcj6.test \
+override-conditional-2.test \
+pr8365-remake-timing.test \
+yacc-dist-nobuild-subdir.test \
+txinfo5.test
+
# Some testsuite-influential variables should be overridable from the
# test scripts, but not from the environment.
-AM_TESTS_ENVIRONMENT = \
- test x"$$me" = x || unset me; \
- test x"$$required" = x || unset required; \
- test x"$$parallel_tests" = x || unset parallel_tests; \
- test x"$$original_AUTOMAKE" = x || unset original_AUTOMAKE; \
- test x"$$original_ACLOCAL" = x || unset original_ACLOCAL;
-
-TESTS = \
+# The `AM_TESTS_REEXEC=no' setting tells the tests not to needlessly
+# re-execute themselves with the shell detected at configure time, since
+# we are already running them under it explicitly in our setup (see e.g.
+# the definition of TEST_LOG_COMPILER above).
+AM_TESTS_ENVIRONMENT = test x"$$me" = x || unset me; test \
+ x"$$required" = x || unset required; test x"$$using_tap" = x \
+ || unset using_tap; test x"$$parallel_tests" = x || unset \
+ parallel_tests; test x"$$test_prefer_config_shell" || unset \
+ test_prefer_config_shell; test x"$$original_AUTOMAKE" = x || \
+ unset original_AUTOMAKE; test x"$$original_ACLOCAL" = x || \
+ unset original_ACLOCAL; AM_TESTS_REEXEC=no; export \
+ AM_TESTS_REEXEC; stderr_fileno_=9; export stderr_fileno_;
+# We want warning messages and explanations for skipped tests to go to
+# the console if possible, so set up `stderr_fileno_' properly.
+AM_TESTS_FD_REDIRECT = 9>&2
+
+# The order here is mostly alphabetical, with the deliberate exception
+# that tests having a high runtime (especially TAP tests that run various
+# checks sequentially) are listed early; this improved performance on
+# concurrent testsuite runs.
+handwritten_TESTS = \
+depmod.tap \
+instspc.tap \
aclocal.test \
aclocal3.test \
aclocal4.test \
acloca21.test \
acloca22.test \
acloca23.test \
-acoutnoq.test \
-acoutpt.test \
-acoutpt2.test \
-acoutqnl.test \
-acoutbs.test \
-acoutbs2.test \
+ac-output-old.tap \
acsilent.test \
acsubst.test \
acsubst2.test \
+add-missing.tap \
all.test \
all2.test \
alloca.test \
autohdr3.test \
autohdr4.test \
autohdrdry.test \
-automake.test \
+automake-cmdline.tap \
auxdir.test \
-auxdir2.test \
-auxdir3.test \
-auxdir4.test \
-auxdir5.test \
auxdir6.test \
auxdir7.test \
auxdir8.test \
-auxdir9.test \
+auxdir-autodetect.test \
+auxdir-computed.tap \
+auxdir-misplaced.test \
+auxdir-nonexistent.test \
+auxdir-unportable.tap \
backcompat.test \
backcompat2.test \
backcompat3.test \
check6.test \
check7.test \
check8.test \
-check9.test \
check10.test \
check11.test \
check12.test \
+check-subst.test \
+check-subst-prog.test \
check-exported-srcdir.test \
+check-fd-redirect.test \
check-tests-in-builddir.test \
-check-tests_environment.test \
+check-no-test-driver.test \
check-concurrency-bug9245.test \
-tests-environment-backcompat.test \
checkall.test \
clean.test \
clean2.test \
condinc.test \
condinc2.test \
condlib.test \
-condman.test \
condman2.test \
condman3.test \
configure.test \
depcomp7.test \
depcomp8a.test \
depcomp8b.test \
-depcomp9.test \
-depcomp10.test \
depdist.test \
depend.test \
depend2.test \
extra10.test \
extra11.test \
extra12.test \
+extra-programs-empty.test \
extradep.test \
extradep2.test \
f90only.test \
help-silent.test \
help-upc.test \
hfs.test \
-hosts.test \
implicit.test \
info.test \
init.test \
instmany.test \
instmany-mans.test \
instmany-python.test \
-instspc-data.test \
-$(instspc_tests) \
interp.test \
interp2.test \
java.test \
lexcpp.test \
lexvpath.test \
lex-subobj-nodep.test \
+lex-lib.test \
+lex-lib-external.test \
+lex-libobj.test \
+lex-noyywrap.test \
lflags.test \
lflags2.test \
libexec.test \
parallel-tests.test \
parallel-tests2.test \
parallel-tests3.test \
-parallel-tests4.test \
parallel-tests5.test \
parallel-tests6.test \
-parallel-tests7.test \
parallel-tests8.test \
parallel-tests9.test \
parallel-tests10.test \
-parallel-tests-am_tests_environment.test \
-parallel-tests-unreadable-log.test \
+parallel-tests-suffix.test \
+parallel-tests-suffix-prog.test \
+parallel-tests-log-compiler-1.test \
+parallel-tests-log-compiler-2.test \
+parallel-tests-dry-run.test \
+parallel-tests-fd-redirect.test \
+parallel-tests-extra-programs.test \
+parallel-tests-unreadable.test \
parallel-tests-subdir.test \
parallel-tests-interrupt.test \
parallel-tests-reset-term.test \
parallel-tests-log-override-1.test \
parallel-tests-log-override-2.test \
parallel-tests-log-override-recheck.test \
-parallel-tests-cmdline-override.test \
parallel-tests-log-compiler-example.test \
+parallel-tests-cmdline-override.test \
parallel-tests-fork-bomb.test \
+parallel-tests-empty-testlogs.test \
+parallel-tests-driver-install.test \
+parallel-tests-no-color-in-log.test \
+parallel-tests-no-spurious-summary.test \
+parallel-tests-exit-statuses.test \
+parallel-tests-console-output.test \
+parallel-tests-once.test \
+tests-environment.test \
+am-tests-environment.test \
+tests-environment-backcompat.test \
+testsuite-summary-color.test \
+testsuite-summary-count.test \
+testsuite-summary-count-many.test \
+testsuite-summary-reference-log.test \
+test-driver-custom-no-extra-driver.test \
+test-driver-custom.test \
+test-driver-custom-xfail-tests.test \
+test-driver-custom-multitest.test \
+test-driver-custom-multitest-recheck.test \
+test-driver-custom-multitest-recheck2.test \
+test-driver-custom-html.test \
+test-driver-custom-no-html.test \
+test-driver-create-log-dir.test \
+test-driver-strip-vpath.test \
+test-driver-trs-suffix-registered.test \
+test-driver-fail.test \
+test-harness-vpath-rewrite.test \
+test-log.test \
+test-logs-repeated.test \
+test-metadata-global-log.test \
+test-metadata-global-result.test \
+test-metadata-recheck.test \
+test-metadata-results.test \
+test-missing.test \
+test-missing2.test \
+test-trs-basic.test \
+test-trs-recover.test \
+test-trs-recover2.test \
parse.test \
percent.test \
percent2.test \
primary-prefix-couples-force-valid.test \
primary-prefix-couples-documented-valid.test \
proginst.test \
+programs-primary-rewritten.test \
python.test \
python2.test \
python3.test \
remake10c.test \
remake11.test \
remake12.test \
+remake-all-1.test \
+remake-all-2.test \
remake-subdir-from-subdir.test \
remake-subdir-gnu.test \
remake-subdir.test \
rulepat.test \
self-check-cleanup.test \
self-check-dir.test \
-self-check-env-sanitize.test \
-self-check-exit.test \
+self-check-env-sanitize.tap \
+self-check-exit.tap \
+self-check-explicit-skips.test \
self-check-is_newest.test \
-self-check-me.test \
+self-check-me.tap \
+self-check-reexec.test \
self-check-report.test \
self-check-sanity.test \
+self-check-seq.test \
+self-check-tap.test \
self-check-unindent.test \
sanity.test \
scripts.test \
silent8.test \
silent9.test \
silentcxx.test \
+silentcxx-gcc.test \
silentf77.test \
silentf90.test \
silent-many-gcc.test \
subpkg3.test \
subpkg4.test \
subst.test \
-subst2.test \
subst3.test \
subst4.test \
subst5.test \
+subst-no-trailing-empty-line.test \
substref.test \
substre2.test \
substtarg.test \
suffix.test \
suffix2.test \
-suffix3.test \
+suffix3.tap \
suffix4.test \
suffix5.test \
suffix6.test \
suffix6b.test \
suffix6c.test \
suffix7.test \
-suffix8.test \
+suffix8.tap \
suffix9.test \
-suffix10.test \
-suffix11.test \
+suffix10.tap \
+suffix11.tap \
suffix12.test \
suffix13.test \
-suffix-chain.test \
+suffix-chain.tap \
symlink.test \
symlink2.test \
syntax.test \
+$(tap_with_common_setup_TESTS) \
+$(tap_other_TESTS) \
tags.test \
tags2.test \
tagsub.test \
yflags-d-false-positives.test \
yflags-force-override.test \
yflags-force-conditional.test \
-yflags-var-expand.test \
-$(parallel_tests)
-
+yflags-var-expand.test
+
+
+# List of tests on TAP support that use the files pre-computed by
+# `tap-common-setup.test', and source the `tap-setup.sh' helper
+# script.
+tap_with_common_setup_TESTS = \
+tap-ambiguous-directive.test \
+tap-autonumber.test \
+tap-bailout.test \
+tap-bailout-and-logging.test \
+tap-bailout-suppress-badexit.test \
+tap-bailout-suppress-later-diagnostic.test \
+tap-bailout-suppress-later-errors.test \
+tap-color.test \
+tap-deps.test \
+tap-diagnostic.test \
+tap-empty-diagnostic.test \
+tap-empty.test \
+tap-escape-directive.test \
+tap-escape-directive-2.test \
+tap-exit.test \
+tap-signal.test \
+tap-fancy.test \
+tap-fancy2.test \
+tap-global-log.test \
+tap-global-result.test \
+tap-html.test \
+tap-log.test \
+tap-msg0-result.test \
+tap-msg0-directive.test \
+tap-msg0-planskip.test \
+tap-msg0-bailout.test \
+tap-msg0-misc.test \
+tap-merge-stdout-stderr.test \
+tap-no-merge-stdout-stderr.test \
+tap-no-disable-hard-error.test \
+tap-no-spurious-summary.test \
+tap-no-spurious-numbers.test \
+tap-no-spurious.test \
+tap-not-ok-skip.test \
+tap-number-wordboundary.test \
+tap-numeric-description.test \
+tap-negative-numbers.test \
+tap-numbers-leading-zero.test \
+tap-out-of-order.test \
+tap-passthrough.test \
+tap-passthrough-exit.test \
+tap-plan.test \
+tap-plan-corner.test \
+tap-plan-errors.test \
+tap-plan-middle.test \
+tap-plan-whitespace.test \
+tap-plan-leading-zero.test \
+tap-plan-malformed.test \
+tap-missing-plan-and-bad-exit.test \
+tap-planskip.test \
+tap-planskip-late.test \
+tap-planskip-and-logging.test \
+tap-planskip-unplanned.test \
+tap-planskip-unplanned-corner.test \
+tap-planskip-case-insensitive.test \
+tap-planskip-whitespace.test \
+tap-planskip-badexit.test \
+tap-planskip-bailout.test \
+tap-planskip-later-errors.test \
+tap-realtime.test \
+tap-test-number-0.test \
+tap-recheck-logs.test \
+tap-result-comment.test \
+tap-todo-skip-together.test \
+tap-todo-skip-whitespace.test \
+tap-todo-skip.test \
+tap-unplanned.test \
+tap-whitespace-normalization.test \
+tap-with-and-without-number.test \
+tap-xfail-tests.test
+
+
+# Other tests on TAP support.
+tap_other_TESTS = \
+tap-common-setup.test \
+tap-bad-prog.tap \
+tap-basic.test \
+tap-diagnostic-custom.test \
+tap-driver-stderr.test \
+tap-doc.test \
+tap-doc2.test \
+tap-more.test \
+tap-more2.test \
+tap-recheck.test \
+tap-summary.test \
+tap-summary-color.test
+
+wrap_TESTS = check-concurrency-bug9245-w.test_pt \
+ check-exported-srcdir-w.test_pt check-fd-redirect-w.test_pt \
+ check-subst-prog-w.test_pt check-subst-w.test_pt \
+ check-tests-in-builddir-w.test_pt check-w.test_pt \
+ check11-w.test_pt check12-w.test_pt check2-w.test_pt \
+ check3-w.test_pt check4-w.test_pt check5-w.test_pt \
+ check6-w.test_pt check7-w.test_pt check8-w.test_pt \
+ color-w.test_pt color2-w.test_pt comment9-w.test_pt \
+ dejagnu-w.test_pt exeext4-w.test_pt maken3-w.test_pt \
+ maken4-w.test_pt posixsubst-tests-w.test_pt \
+ tests-environment-w.test_pt ar-lib-w.test_cs compile-w.test_cs \
+ compile2-w.test_cs compile3-w.test_cs compile4-w.test_cs \
+ compile5-w.test_cs compile6-w.test_cs instsh2-w.test_cs \
+ instsh3-w.test_cs mdate5-w.test_cs mdate6-w.test_cs \
+ missing-w.test_cs missing2-w.test_cs missing3-w.test_cs \
+ missing5-w.test_cs mkinst3-w.test_cs \
+ tap-ambiguous-directive-w.test_pltap \
+ tap-autonumber-w.test_pltap tap-bad-prog-w.tap_pltap \
+ tap-bailout-and-logging-w.test_pltap \
+ tap-bailout-suppress-badexit-w.test_pltap \
+ tap-bailout-suppress-later-diagnostic-w.test_pltap \
+ tap-bailout-suppress-later-errors-w.test_pltap \
+ tap-bailout-w.test_pltap tap-basic-w.test_pltap \
+ tap-color-w.test_pltap tap-deps-w.test_pltap \
+ tap-diagnostic-custom-w.test_pltap tap-diagnostic-w.test_pltap \
+ tap-doc-w.test_pltap tap-driver-stderr-w.test_pltap \
+ tap-empty-diagnostic-w.test_pltap tap-empty-w.test_pltap \
+ tap-escape-directive-2-w.test_pltap \
+ tap-escape-directive-w.test_pltap tap-exit-w.test_pltap \
+ tap-fancy-w.test_pltap tap-fancy2-w.test_pltap \
+ tap-global-log-w.test_pltap tap-global-result-w.test_pltap \
+ tap-html-w.test_pltap tap-log-w.test_pltap \
+ tap-merge-stdout-stderr-w.test_pltap \
+ tap-missing-plan-and-bad-exit-w.test_pltap \
+ tap-more-w.test_pltap tap-more2-w.test_pltap \
+ tap-msg0-bailout-w.test_pltap tap-msg0-directive-w.test_pltap \
+ tap-msg0-misc-w.test_pltap tap-msg0-planskip-w.test_pltap \
+ tap-msg0-result-w.test_pltap tap-negative-numbers-w.test_pltap \
+ tap-no-disable-hard-error-w.test_pltap \
+ tap-no-merge-stdout-stderr-w.test_pltap \
+ tap-no-spurious-numbers-w.test_pltap \
+ tap-no-spurious-summary-w.test_pltap \
+ tap-no-spurious-w.test_pltap tap-not-ok-skip-w.test_pltap \
+ tap-number-wordboundary-w.test_pltap \
+ tap-numbers-leading-zero-w.test_pltap \
+ tap-numeric-description-w.test_pltap \
+ tap-out-of-order-w.test_pltap \
+ tap-passthrough-exit-w.test_pltap tap-passthrough-w.test_pltap \
+ tap-plan-corner-w.test_pltap tap-plan-errors-w.test_pltap \
+ tap-plan-leading-zero-w.test_pltap \
+ tap-plan-malformed-w.test_pltap tap-plan-middle-w.test_pltap \
+ tap-plan-whitespace-w.test_pltap tap-plan-w.test_pltap \
+ tap-planskip-and-logging-w.test_pltap \
+ tap-planskip-badexit-w.test_pltap \
+ tap-planskip-bailout-w.test_pltap \
+ tap-planskip-case-insensitive-w.test_pltap \
+ tap-planskip-late-w.test_pltap \
+ tap-planskip-later-errors-w.test_pltap \
+ tap-planskip-unplanned-corner-w.test_pltap \
+ tap-planskip-unplanned-w.test_pltap \
+ tap-planskip-whitespace-w.test_pltap tap-planskip-w.test_pltap \
+ tap-realtime-w.test_pltap tap-recheck-logs-w.test_pltap \
+ tap-recheck-w.test_pltap tap-result-comment-w.test_pltap \
+ tap-signal-w.test_pltap tap-test-number-0-w.test_pltap \
+ tap-todo-skip-together-w.test_pltap \
+ tap-todo-skip-whitespace-w.test_pltap \
+ tap-todo-skip-w.test_pltap tap-unplanned-w.test_pltap \
+ tap-whitespace-normalization-w.test_pltap \
+ tap-with-and-without-number-w.test_pltap \
+ tap-xfail-tests-w.test_pltap
+wrap_LOGS = check-concurrency-bug9245-w.log \
+ check-exported-srcdir-w.log check-fd-redirect-w.log \
+ check-subst-prog-w.log check-subst-w.log \
+ check-tests-in-builddir-w.log check-w.log check11-w.log \
+ check12-w.log check2-w.log check3-w.log check4-w.log \
+ check5-w.log check6-w.log check7-w.log check8-w.log \
+ color-w.log color2-w.log comment9-w.log dejagnu-w.log \
+ exeext4-w.log maken3-w.log maken4-w.log posixsubst-tests-w.log \
+ tests-environment-w.log ar-lib-w.log compile-w.log \
+ compile2-w.log compile3-w.log compile4-w.log compile5-w.log \
+ compile6-w.log instsh2-w.log instsh3-w.log mdate5-w.log \
+ mdate6-w.log missing-w.log missing2-w.log missing3-w.log \
+ missing5-w.log mkinst3-w.log tap-ambiguous-directive-w.log \
+ tap-autonumber-w.log tap-bad-prog-w.log \
+ tap-bailout-and-logging-w.log \
+ tap-bailout-suppress-badexit-w.log \
+ tap-bailout-suppress-later-diagnostic-w.log \
+ tap-bailout-suppress-later-errors-w.log tap-bailout-w.log \
+ tap-basic-w.log tap-color-w.log tap-deps-w.log \
+ tap-diagnostic-custom-w.log tap-diagnostic-w.log tap-doc-w.log \
+ tap-driver-stderr-w.log tap-empty-diagnostic-w.log \
+ tap-empty-w.log tap-escape-directive-2-w.log \
+ tap-escape-directive-w.log tap-exit-w.log tap-fancy-w.log \
+ tap-fancy2-w.log tap-global-log-w.log tap-global-result-w.log \
+ tap-html-w.log tap-log-w.log tap-merge-stdout-stderr-w.log \
+ tap-missing-plan-and-bad-exit-w.log tap-more-w.log \
+ tap-more2-w.log tap-msg0-bailout-w.log \
+ tap-msg0-directive-w.log tap-msg0-misc-w.log \
+ tap-msg0-planskip-w.log tap-msg0-result-w.log \
+ tap-negative-numbers-w.log tap-no-disable-hard-error-w.log \
+ tap-no-merge-stdout-stderr-w.log tap-no-spurious-numbers-w.log \
+ tap-no-spurious-summary-w.log tap-no-spurious-w.log \
+ tap-not-ok-skip-w.log tap-number-wordboundary-w.log \
+ tap-numbers-leading-zero-w.log tap-numeric-description-w.log \
+ tap-out-of-order-w.log tap-passthrough-exit-w.log \
+ tap-passthrough-w.log tap-plan-corner-w.log \
+ tap-plan-errors-w.log tap-plan-leading-zero-w.log \
+ tap-plan-malformed-w.log tap-plan-middle-w.log \
+ tap-plan-whitespace-w.log tap-plan-w.log \
+ tap-planskip-and-logging-w.log tap-planskip-badexit-w.log \
+ tap-planskip-bailout-w.log tap-planskip-case-insensitive-w.log \
+ tap-planskip-late-w.log tap-planskip-later-errors-w.log \
+ tap-planskip-unplanned-corner-w.log \
+ tap-planskip-unplanned-w.log tap-planskip-whitespace-w.log \
+ tap-planskip-w.log tap-realtime-w.log tap-recheck-logs-w.log \
+ tap-recheck-w.log tap-result-comment-w.log tap-signal-w.log \
+ tap-test-number-0-w.log tap-todo-skip-together-w.log \
+ tap-todo-skip-whitespace-w.log tap-todo-skip-w.log \
+ tap-unplanned-w.log tap-whitespace-normalization-w.log \
+ tap-with-and-without-number-w.log tap-xfail-tests-w.log
+TEST_PT_LOG_DRIVER = \
+ $(TEST_LOG_DRIVER)
+
+AM_TEST_PT_LOG_DRIVER_FLAGS = \
+ $(AM_TEST_LOG_DRIVER_FLAGS) \
+ $(TEST_LOG_DRIVER_FLAGS)
+
+TEST_PT_LOG_COMPILER = \
+ $(LOG_COMPILER) $(srcdir)/wrap-tests.sh \
+ --type pt --define parallel_tests "yes"
+
+TEST_CS_LOG_DRIVER = \
+ $(TEST_LOG_DRIVER)
+
+AM_TEST_CS_LOG_DRIVER_FLAGS = \
+ $(AM_TEST_LOG_DRIVER_FLAGS) \
+ $(TEST_LOG_DRIVER_FLAGS)
+
+TEST_CS_LOG_COMPILER = \
+ $(LOG_COMPILER) $(srcdir)/wrap-tests.sh \
+ --type cs --define test_prefer_config_shell "yes"
+
+TEST_PLTAP_LOG_DRIVER = \
+ $(TEST_LOG_DRIVER)
+
+AM_TEST_PLTAP_LOG_DRIVER_FLAGS = \
+ $(AM_TEST_LOG_DRIVER_FLAGS) \
+ $(TEST_LOG_DRIVER_FLAGS)
+
+TEST_PLTAP_LOG_COMPILER = \
+ $(LOG_COMPILER) $(srcdir)/wrap-tests.sh \
+ --type pltap --define am_tap_implementation "perl"
+
+TAP_PLTAP_LOG_DRIVER = \
+ $(TAP_LOG_DRIVER)
+
+AM_TAP_PLTAP_LOG_DRIVER_FLAGS = \
+ $(AM_TAP_LOG_DRIVER_FLAGS) \
+ $(TAP_LOG_DRIVER_FLAGS)
+
+TAP_PLTAP_LOG_COMPILER = \
+ $(LOG_COMPILER) $(srcdir)/wrap-tests.sh \
+ --type pltap --define am_tap_implementation "perl"
+
+checked_test_extensions = .test .tap
+expected_list_of_tests = $(handwritten_TESTS)
+am__tmk = tests-in-makefile-list.tmp
+am__tfs = tests-on-filesystem-list.tmp
+am__tdf = diff-in-tests-lists.tmp
all: all-am
.SUFFIXES:
-.SUFFIXES: .html .log .test
-$(srcdir)/Makefile.in: $(srcdir)/Makefile.am $(srcdir)/parallel-tests.am $(srcdir)/instspc-tests.am $(am__configure_deps)
+.SUFFIXES: .html .log .tap .tap_pltap .test .test_cs .test_pltap .test_pt .trs
+$(srcdir)/Makefile.in: $(srcdir)/Makefile.am $(srcdir)/list-of-tests.mk $(srcdir)/wrap-tests.am $(top_srcdir)/CheckListOfTests.am $(am__configure_deps)
@for dep in $?; do \
case '$(am__configure_deps)' in \
*$$dep*) \
cscope cscopelist:
+# Recover from deleted `.trs' file; this should ensure that
+# "rm -f foo.log; make foo.trs" re-run `foo.test', and re-create
+# both `foo.log' and `foo.trs'. Break the recipe in two subshells
+# to avoid problems with "make -n".
+.log.trs:
+ rm -f $< $@
+ $(MAKE) $(AM_MAKEFLAGS) $<
+
+# Helper recipe used by $(TEST_SUITE_LOG) below, to avoid problems with
+# "make -n". Break this recipe in multiple shell invocations too, to
+# really work as expected with "make -n".
+am--redo-logs:
+ @rm -f $$redo_logs
+ @rm -f $$redo_results
+ @if test -n "$$am__remaking_logs"; then \
+ echo "fatal: making $(TEST_SUITE_LOG): possible infinite" \
+ "recursion detected" >&2; \
+ else \
+ am__remaking_logs=yes $(MAKE) $(AM_MAKEFLAGS) $$redo_logs; \
+ fi;
+ @st=0; \
+ errmsg="fatal: making $(TEST_SUITE_LOG): failed to create"; \
+ for i in $$redo_bases; do \
+ test -f $$i.trs && test -r $$i.trs \
+ || { echo "$$errmsg $$i.trs"; st=1; }; \
+ test -f $$i.log && test -r $$i.log \
+ || { echo "$$errmsg $$i.log"; st=1; }; \
+ done; \
+ test $$st -eq 0
+
$(TEST_SUITE_LOG): $(TEST_LOGS)
- @$(am__sh_e_setup); \
- list='$(TEST_LOGS)'; \
- results=`for f in $$list; do \
- test -r $$f && read line < $$f && echo "$$line" \
- || echo FAIL; \
- done`; \
- all=`echo "$$results" | sed '/^$$/d' | wc -l | sed -e 's/^[ ]*//'`; \
- fail=`echo "$$results" | grep -c '^FAIL'`; \
- pass=`echo "$$results" | grep -c '^PASS'`; \
- skip=`echo "$$results" | grep -c '^SKIP'`; \
- xfail=`echo "$$results" | grep -c '^XFAIL'`; \
- xpass=`echo "$$results" | grep -c '^XPASS'`; \
- failures=`expr $$fail + $$xpass`; \
- all=`expr $$all - $$skip`; \
- if test "$$all" -eq 1; then tests=test; All=; \
- else tests=tests; All="All "; fi; \
- case fail=$$fail:xpass=$$xpass:xfail=$$xfail in \
- fail=0:xpass=0:xfail=0) \
- msg="$$All$$all $$tests passed. "; \
- exit=true;; \
- fail=0:xpass=0:xfail=*) \
- msg="$$All$$all $$tests behaved as expected"; \
- if test "$$xfail" -eq 1; then xfailures=failure; \
- else xfailures=failures; fi; \
- msg="$$msg ($$xfail expected $$xfailures). "; \
- exit=true;; \
- fail=*:xpass=0:xfail=*) \
- msg="$$fail of $$all $$tests failed. "; \
- exit=false;; \
- fail=*:xpass=*:xfail=*) \
- msg="$$failures of $$all $$tests did not behave as expected"; \
- if test "$$xpass" -eq 1; then xpasses=pass; \
- else xpasses=passes; fi; \
- msg="$$msg ($$xpass unexpected $$xpasses). "; \
- exit=false;; \
- *) \
- echo >&2 "incorrect case"; exit 4;; \
- esac; \
- if test "$$skip" -ne 0; then \
- if test "$$skip" -eq 1; then \
- msg="$$msg($$skip test was not run). "; \
- else \
- msg="$$msg($$skip tests were not run). "; \
- fi; \
- fi; \
+ @$(am__set_TESTS_bases); \
+ am__f_ok () { test -f "$$1" && test -r "$$1"; }; \
+ redo_bases=`for i in $$bases; do \
+ am__f_ok $$i.trs && am__f_ok $$i.log || echo $$i; \
+ done`; \
+ if test -n "$$redo_bases"; then \
+ redo_logs=`for i in $$redo_bases; do echo $$i.log; done`; \
+ redo_results=`for i in $$redo_bases; do echo $$i.trs; done`; \
+ redo_bases="$$redo_bases" \
+ redo_logs="$$redo_logs" \
+ redo_results="$$redo_results" \
+ $(MAKE) $(AM_MAKEFLAGS) am--redo-logs || exit 1; \
+ else :; fi;
+ @$(am__sh_e_setup); $(am__tty_colors); $(am__set_TESTS_bases); \
+ ws='[ ]'; \
+ results=`for b in $$bases; do echo $$b.trs; done`; \
+ test -n "$$results" || results=/dev/null; \
+ all=` grep "^$$ws*:test-result:" $$results | wc -l`; \
+ pass=` grep "^$$ws*:test-result:$$ws*PASS" $$results | wc -l`; \
+ fail=` grep "^$$ws*:test-result:$$ws*FAIL" $$results | wc -l`; \
+ skip=` grep "^$$ws*:test-result:$$ws*SKIP" $$results | wc -l`; \
+ xfail=`grep "^$$ws*:test-result:$$ws*XFAIL" $$results | wc -l`; \
+ xpass=`grep "^$$ws*:test-result:$$ws*XPASS" $$results | wc -l`; \
+ error=`grep "^$$ws*:test-result:$$ws*ERROR" $$results | wc -l`; \
+ if test `expr $$fail + $$xpass + $$error` -eq 0; then \
+ success=true; \
+ else \
+ success=false; \
+ fi; \
+ br='==================='; br=$$br$$br$$br$$br; \
+ result_count () \
+ { \
+ if test x"$$1" = x"--maybe-color"; then \
+ maybe_colorize=yes; \
+ elif test x"$$1" = x"--no-color"; then \
+ maybe_colorize=no; \
+ else \
+ echo "$@: invalid 'result_count' usage" >&2; exit 4; \
+ fi; \
+ shift; \
+ desc=$$1 count=$$2; \
+ if test $$maybe_colorize = yes && test $$count -gt 0; then \
+ color_start=$$3 color_end=$$std; \
+ else \
+ color_start= color_end=; \
+ fi; \
+ echo "$${color_start}# $$desc $$count$${color_end}"; \
+ }; \
+ create_testsuite_report () \
+ { \
+ result_count $$1 "TOTAL:" $$all "$$brg"; \
+ result_count $$1 "PASS: " $$pass "$$grn"; \
+ result_count $$1 "SKIP: " $$skip "$$blu"; \
+ result_count $$1 "XFAIL:" $$xfail "$$lgn"; \
+ result_count $$1 "FAIL: " $$fail "$$red"; \
+ result_count $$1 "XPASS:" $$xpass "$$red"; \
+ result_count $$1 "ERROR:" $$error "$$mgn"; \
+ }; \
{ \
echo "$(PACKAGE_STRING): $(subdir)/$(TEST_SUITE_LOG)" | \
$(am__rst_title); \
- echo "$$msg"; \
+ create_testsuite_report --no-color; \
echo; \
echo ".. contents:: :depth: 2"; \
echo; \
- for f in $$list; do \
- test -r $$f && read line < $$f || line=; \
- case $$line in \
- PASS:*|XFAIL:*);; \
- *) echo; cat $$f;; \
- esac; \
- done; \
+ for i in $$bases; do \
+ if grep "^$$ws*:copy-in-global-log:$$ws*no$$ws*$$" $$i.trs \
+ >/dev/null; then continue; \
+ fi; \
+ glob_res=`sed -n -e "s/$$ws*$$//" \
+ -e "s/^$$ws*:global-test-result:$$ws*//p" \
+ $$i.trs`; \
+ test -n "$$glob_res" || glob_res=RUN; \
+ echo "$$glob_res: $$i" | $(am__rst_section); \
+ if test ! -r $$i.log; then \
+ echo "fatal: making $@: $$i.log is unreadable" >&2; \
+ exit 1; \
+ fi; \
+ cat $$i.log; echo; \
+ done; \
} >$(TEST_SUITE_LOG).tmp; \
mv $(TEST_SUITE_LOG).tmp $(TEST_SUITE_LOG); \
- if test "$$failures" -ne 0; then \
- msg="$${msg}See $(subdir)/$(TEST_SUITE_LOG). "; \
- if test -n "$(PACKAGE_BUGREPORT)"; then \
- msg="$${msg}Please report to $(PACKAGE_BUGREPORT). "; \
- fi; \
- fi; \
- test x"$$VERBOSE" = x || $$exit || cat $(TEST_SUITE_LOG); \
- $(am__tty_colors); \
- if $$exit; then \
+ if $$success; then \
col="$$grn"; \
else \
col="$$red"; \
+ test x"$$VERBOSE" = x || cat $(TEST_SUITE_LOG); \
+ fi; \
+ echo "$${col}$$br$${std}"; \
+ echo "$${col}Testsuite summary for $(PACKAGE_STRING)$${std}"; \
+ echo "$${col}$$br$${std}"; \
+ create_testsuite_report --maybe-color; \
+ echo "$$col$$br$$std"; \
+ if $$success; then :; else \
+ echo "$${col}See $(subdir)/$(TEST_SUITE_LOG)$${std}"; \
+ if test -n "$(PACKAGE_BUGREPORT)"; then \
+ echo "$${col}Please report to $(PACKAGE_BUGREPORT)$${std}"; \
+ fi; \
+ echo "$$col$$br$$std"; \
fi; \
- echo "$$msg" | $(am__text_box) "col=$$col" "std=$$std"; \
- $$exit || exit 1
+ $$success || exit 1
# Run all the tests.
check-TESTS:
@list='$(RECHECK_LOGS)'; test -z "$$list" || rm -f $$list
+ @list='$(RECHECK_LOGS:.log=.trs)'; test -z "$$list" || rm -f $$list
@test -z "$(TEST_SUITE_LOG)" || rm -f $(TEST_SUITE_LOG)
- @list='$(TEST_LOGS)'; \
- list=`for f in $$list; do \
- test .log = $$f || echo $$f; \
- done | tr '\012\015' ' '`; \
+ @list='$(TEST_LOGS)'; \
+ list=`for i in $$list; do \
+ test .log = $$i || echo $$i; \
+ done | tr '\012\015' ' '`; \
+ list=`echo "$$list" | sed 's/ *$$//'`; \
$(MAKE) $(AM_MAKEFLAGS) $(TEST_SUITE_LOG) TEST_LOGS="$$list"
.log.html:
$(MAKE) $(AM_MAKEFLAGS) $(TEST_SUITE_HTML) || exit 4; \
exit $$rv
recheck recheck-html:
- @target=`echo $@ | sed 's,^re,,'`; \
- list='$(TEST_LOGS)'; \
- list=`for f in $$list; do \
- test -f $$f || continue; \
- if test -r $$f && read line < $$f; then \
- case $$line in FAIL*|XPASS*) echo $$f;; esac; \
- else echo $$f; fi; \
+ @ws='[ ]'; \
+ target=`echo $@ | sed 's,^re,,'`; \
+ $(am__set_TESTS_bases); \
+ list=`for i in $$bases; do \
+ test -f $$i.trs || test -f $$i.log || continue; \
+ grep "^$$ws*:recheck:$$ws*no$$ws*$$" $$i.trs \
+ >/dev/null 2>&1 || echo $$i.log; \
done | tr '\012\015' ' '`; \
list=`echo "$$list" | sed 's/ *$$//'`; \
$(MAKE) $(AM_MAKEFLAGS) $$target AM_MAKEFLAGS='$(AM_MAKEFLAGS) TEST_LOGS="'"$$list"'"'
.test.log:
- @p='$<'; $(am__check_pre) $(TEST_LOG_COMPILE) "$$tst" $(am__check_post)
+ @p='$<'; $(am__check_pre) $(TEST_LOG_DRIVER) --test-name "$$f" \
+ --log-file '$*.log' --trs-file '$*.trs' \
+ $(am__common_driver_flags) $(AM_TEST_LOG_DRIVER_FLAGS) $(TEST_LOG_DRIVER_FLAGS) -- $(TEST_LOG_COMPILE) "$$tst" \
+ $(AM_TESTS_FD_REDIRECT)
+.tap.log:
+ @p='$<'; $(am__check_pre) $(TAP_LOG_DRIVER) --test-name "$$f" \
+ --log-file '$*.log' --trs-file '$*.trs' \
+ $(am__common_driver_flags) $(AM_TAP_LOG_DRIVER_FLAGS) $(TAP_LOG_DRIVER_FLAGS) -- $(TAP_LOG_COMPILE) "$$tst" \
+ $(AM_TESTS_FD_REDIRECT)
+.test_pt.log:
+ @p='$<'; $(am__check_pre) $(TEST_PT_LOG_DRIVER) --test-name "$$f" \
+ --log-file '$*.log' --trs-file '$*.trs' \
+ $(am__common_driver_flags) $(AM_TEST_PT_LOG_DRIVER_FLAGS) $(TEST_PT_LOG_DRIVER_FLAGS) -- $(TEST_PT_LOG_COMPILE) "$$tst" \
+ $(AM_TESTS_FD_REDIRECT)
+.test_cs.log:
+ @p='$<'; $(am__check_pre) $(TEST_CS_LOG_DRIVER) --test-name "$$f" \
+ --log-file '$*.log' --trs-file '$*.trs' \
+ $(am__common_driver_flags) $(AM_TEST_CS_LOG_DRIVER_FLAGS) $(TEST_CS_LOG_DRIVER_FLAGS) -- $(TEST_CS_LOG_COMPILE) "$$tst" \
+ $(AM_TESTS_FD_REDIRECT)
+.test_pltap.log:
+ @p='$<'; $(am__check_pre) $(TEST_PLTAP_LOG_DRIVER) --test-name "$$f" \
+ --log-file '$*.log' --trs-file '$*.trs' \
+ $(am__common_driver_flags) $(AM_TEST_PLTAP_LOG_DRIVER_FLAGS) $(TEST_PLTAP_LOG_DRIVER_FLAGS) -- $(TEST_PLTAP_LOG_COMPILE) "$$tst" \
+ $(AM_TESTS_FD_REDIRECT)
+.tap_pltap.log:
+ @p='$<'; $(am__check_pre) $(TAP_PLTAP_LOG_DRIVER) --test-name "$$f" \
+ --log-file '$*.log' --trs-file '$*.trs' \
+ $(am__common_driver_flags) $(AM_TAP_PLTAP_LOG_DRIVER_FLAGS) $(TAP_PLTAP_LOG_DRIVER_FLAGS) -- $(TAP_PLTAP_LOG_COMPILE) "$$tst" \
+ $(AM_TESTS_FD_REDIRECT)
distdir: $(DISTFILES)
@srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \
fi; \
done
check-am: all-am
- $(MAKE) $(AM_MAKEFLAGS) check-TESTS
+ $(MAKE) $(AM_MAKEFLAGS) check-TESTS check-local
check: check-am
all-am: Makefile
installdirs:
fi
mostlyclean-generic:
-test -z "$(TEST_LOGS)" || rm -f $(TEST_LOGS)
- -test -z "$(TEST_LOGS_TMP)" || rm -f $(TEST_LOGS_TMP)
+ -test -z "$(TEST_LOGS:.log=.trs)" || rm -f $(TEST_LOGS:.log=.trs)
-test -z "$(TEST_SUITE_HTML)" || rm -f $(TEST_SUITE_HTML)
-test -z "$(TEST_SUITE_LOG)" || rm -f $(TEST_SUITE_LOG)
maintainer-clean-generic:
@echo "This command is intended for maintainers to use"
@echo "it deletes files that may require special tools to rebuild."
- -test -z "$(MAINTAINERCLEANFILES)" || rm -f $(MAINTAINERCLEANFILES)
clean: clean-am
clean-am: clean-generic clean-local mostlyclean-am
.MAKE: check-am check-html install-am install-strip recheck \
recheck-html
-.PHONY: all all-am check check-TESTS check-am check-html clean \
- clean-generic clean-local distclean distclean-generic distdir \
- dvi dvi-am html html-am info info-am install install-am \
- install-data install-data-am install-dvi install-dvi-am \
- install-exec install-exec-am install-html install-html-am \
- install-info install-info-am install-man install-pdf \
- install-pdf-am install-ps install-ps-am install-strip \
- installcheck installcheck-am installdirs maintainer-clean \
- maintainer-clean-generic mostlyclean mostlyclean-generic pdf \
- pdf-am ps ps-am recheck recheck-html uninstall uninstall-am
-
-backcompat5-p.log: backcompat5.test
-check-concurrency-bug9245-p.log: check-concurrency-bug9245.test
-check-exported-srcdir-p.log: check-exported-srcdir.test
-check-tests-in-builddir-p.log: check-tests-in-builddir.test
-check-tests_environment-p.log: check-tests_environment.test
-check-p.log: check.test
-check10-p.log: check10.test
-check11-p.log: check11.test
-check12-p.log: check12.test
-check2-p.log: check2.test
-check3-p.log: check3.test
-check4-p.log: check4.test
-check5-p.log: check5.test
-check6-p.log: check6.test
-check7-p.log: check7.test
-check8-p.log: check8.test
-check9-p.log: check9.test
-color-p.log: color.test
-color2-p.log: color2.test
-comment9-p.log: comment9.test
-dejagnu-p.log: dejagnu.test
-exeext4-p.log: exeext4.test
-maken3-p.log: maken3.test
-maken4-p.log: maken4.test
-posixsubst-tests-p.log: posixsubst-tests.test
-
-$(srcdir)/parallel-tests.am: gen-parallel-tests Makefile.am
- $(AM_V_GEN)($(am__cd) $(srcdir) && $(SHELL) ./gen-parallel-tests) >$@
-
-$(parallel_tests): Makefile.am
- $(AM_V_at)rm -f $@ $@-t
- $(AM_V_GEN)input=`echo $@ | sed 's,.*/,,; s,-p.test$$,.test,'`; \
- { echo '#!/bin/sh'; \
- echo '# DO NOT EDIT! GENERATED AUTOMATICALLY!'; \
- echo; \
- echo '# Ensure proper definition of $$testsrcdir.'; \
- echo '. ./defs-static || exit 99'; \
- echo 'test -n "$$testsrcdir" || exit 99 # sanity check'; \
- echo; \
- echo "# Run the test with Automake's parallel-tests driver enabled."; \
- echo 'parallel_tests=yes'; \
- echo "# In the spirit of VPATH, we prefer a test in the build tree"; \
- echo "# over one in the source tree."; \
- echo "if test -f \"./$$input\"; then"; \
- echo " . \"./$$input\""; \
- echo 'else'; \
- echo " . \"\$$testsrcdir/$$input\""; \
- echo 'fi'; \
- } > $@-t
- $(AM_V_at)chmod a+rx $@-t && mv -f $@-t $@
-
-$(srcdir)/instspc-tests.am: instspc-tests.sh Makefile.am
+.PHONY: all all-am am--redo-logs check check-TESTS check-am check-html \
+ check-local clean clean-generic clean-local distclean \
+ distclean-generic distdir dvi dvi-am html html-am info info-am \
+ install install-am install-data install-data-am install-dvi \
+ install-dvi-am install-exec install-exec-am install-html \
+ install-html-am install-info install-info-am install-man \
+ install-pdf install-pdf-am install-ps install-ps-am \
+ install-strip installcheck installcheck-am installdirs \
+ maintainer-clean maintainer-clean-generic mostlyclean \
+ mostlyclean-generic pdf pdf-am ps ps-am recheck recheck-html \
+ uninstall uninstall-am
+
+
+print-list-of-tests:
+ @echo $(handwritten_TESTS)
+.PHONY: print-list-of-tests
+check-concurrency-bug9245-w.log: check-concurrency-bug9245.test
+check-exported-srcdir-w.log: check-exported-srcdir.test
+check-fd-redirect-w.log: check-fd-redirect.test
+check-subst-prog-w.log: check-subst-prog.test
+check-subst-w.log: check-subst.test
+check-tests-in-builddir-w.log: check-tests-in-builddir.test
+check-w.log: check.test
+check11-w.log: check11.test
+check12-w.log: check12.test
+check2-w.log: check2.test
+check3-w.log: check3.test
+check4-w.log: check4.test
+check5-w.log: check5.test
+check6-w.log: check6.test
+check7-w.log: check7.test
+check8-w.log: check8.test
+color-w.log: color.test
+color2-w.log: color2.test
+comment9-w.log: comment9.test
+dejagnu-w.log: dejagnu.test
+exeext4-w.log: exeext4.test
+maken3-w.log: maken3.test
+maken4-w.log: maken4.test
+posixsubst-tests-w.log: posixsubst-tests.test
+tests-environment-w.log: tests-environment.test
+ar-lib-w.log: ar-lib.test
+compile-w.log: compile.test
+compile2-w.log: compile2.test
+compile3-w.log: compile3.test
+compile4-w.log: compile4.test
+compile5-w.log: compile5.test
+compile6-w.log: compile6.test
+instsh2-w.log: instsh2.test
+instsh3-w.log: instsh3.test
+mdate5-w.log: mdate5.test
+mdate6-w.log: mdate6.test
+missing-w.log: missing.test
+missing2-w.log: missing2.test
+missing3-w.log: missing3.test
+missing5-w.log: missing5.test
+mkinst3-w.log: mkinst3.test
+tap-ambiguous-directive-w.log: tap-ambiguous-directive.test
+tap-autonumber-w.log: tap-autonumber.test
+tap-bad-prog-w.log: tap-bad-prog.tap
+tap-bailout-and-logging-w.log: tap-bailout-and-logging.test
+tap-bailout-suppress-badexit-w.log: tap-bailout-suppress-badexit.test
+tap-bailout-suppress-later-diagnostic-w.log: tap-bailout-suppress-later-diagnostic.test
+tap-bailout-suppress-later-errors-w.log: tap-bailout-suppress-later-errors.test
+tap-bailout-w.log: tap-bailout.test
+tap-basic-w.log: tap-basic.test
+tap-color-w.log: tap-color.test
+tap-deps-w.log: tap-deps.test
+tap-diagnostic-custom-w.log: tap-diagnostic-custom.test
+tap-diagnostic-w.log: tap-diagnostic.test
+tap-doc-w.log: tap-doc.test
+tap-driver-stderr-w.log: tap-driver-stderr.test
+tap-empty-diagnostic-w.log: tap-empty-diagnostic.test
+tap-empty-w.log: tap-empty.test
+tap-escape-directive-2-w.log: tap-escape-directive-2.test
+tap-escape-directive-w.log: tap-escape-directive.test
+tap-exit-w.log: tap-exit.test
+tap-fancy-w.log: tap-fancy.test
+tap-fancy2-w.log: tap-fancy2.test
+tap-global-log-w.log: tap-global-log.test
+tap-global-result-w.log: tap-global-result.test
+tap-html-w.log: tap-html.test
+tap-log-w.log: tap-log.test
+tap-merge-stdout-stderr-w.log: tap-merge-stdout-stderr.test
+tap-missing-plan-and-bad-exit-w.log: tap-missing-plan-and-bad-exit.test
+tap-more-w.log: tap-more.test
+tap-more2-w.log: tap-more2.test
+tap-msg0-bailout-w.log: tap-msg0-bailout.test
+tap-msg0-directive-w.log: tap-msg0-directive.test
+tap-msg0-misc-w.log: tap-msg0-misc.test
+tap-msg0-planskip-w.log: tap-msg0-planskip.test
+tap-msg0-result-w.log: tap-msg0-result.test
+tap-negative-numbers-w.log: tap-negative-numbers.test
+tap-no-disable-hard-error-w.log: tap-no-disable-hard-error.test
+tap-no-merge-stdout-stderr-w.log: tap-no-merge-stdout-stderr.test
+tap-no-spurious-numbers-w.log: tap-no-spurious-numbers.test
+tap-no-spurious-summary-w.log: tap-no-spurious-summary.test
+tap-no-spurious-w.log: tap-no-spurious.test
+tap-not-ok-skip-w.log: tap-not-ok-skip.test
+tap-number-wordboundary-w.log: tap-number-wordboundary.test
+tap-numbers-leading-zero-w.log: tap-numbers-leading-zero.test
+tap-numeric-description-w.log: tap-numeric-description.test
+tap-out-of-order-w.log: tap-out-of-order.test
+tap-passthrough-exit-w.log: tap-passthrough-exit.test
+tap-passthrough-w.log: tap-passthrough.test
+tap-plan-corner-w.log: tap-plan-corner.test
+tap-plan-errors-w.log: tap-plan-errors.test
+tap-plan-leading-zero-w.log: tap-plan-leading-zero.test
+tap-plan-malformed-w.log: tap-plan-malformed.test
+tap-plan-middle-w.log: tap-plan-middle.test
+tap-plan-whitespace-w.log: tap-plan-whitespace.test
+tap-plan-w.log: tap-plan.test
+tap-planskip-and-logging-w.log: tap-planskip-and-logging.test
+tap-planskip-badexit-w.log: tap-planskip-badexit.test
+tap-planskip-bailout-w.log: tap-planskip-bailout.test
+tap-planskip-case-insensitive-w.log: tap-planskip-case-insensitive.test
+tap-planskip-late-w.log: tap-planskip-late.test
+tap-planskip-later-errors-w.log: tap-planskip-later-errors.test
+tap-planskip-unplanned-corner-w.log: tap-planskip-unplanned-corner.test
+tap-planskip-unplanned-w.log: tap-planskip-unplanned.test
+tap-planskip-whitespace-w.log: tap-planskip-whitespace.test
+tap-planskip-w.log: tap-planskip.test
+tap-realtime-w.log: tap-realtime.test
+tap-recheck-logs-w.log: tap-recheck-logs.test
+tap-recheck-w.log: tap-recheck.test
+tap-result-comment-w.log: tap-result-comment.test
+tap-signal-w.log: tap-signal.test
+tap-test-number-0-w.log: tap-test-number-0.test
+tap-todo-skip-together-w.log: tap-todo-skip-together.test
+tap-todo-skip-whitespace-w.log: tap-todo-skip-whitespace.test
+tap-todo-skip-w.log: tap-todo-skip.test
+tap-unplanned-w.log: tap-unplanned.test
+tap-whitespace-normalization-w.log: tap-whitespace-normalization.test
+tap-with-and-without-number-w.log: tap-with-and-without-number.test
+tap-xfail-tests-w.log: tap-xfail-tests.test
+
+$(srcdir)/wrap-tests.am: gen-wrap-tests list-of-tests.mk Makefile.am
$(AM_V_GEN)($(am__cd) $(srcdir) \
- && $(SHELL) ./instspc-tests.sh --generate-makefile) >$@
-
-$(instspc_tests): Makefile.am
- $(AM_V_at)rm -f $@ $@-t
- $(AM_V_GEN) :; \
- base=`expr 'x/$@' : 'x.*/instspc-\(.*\)\.test$$'`; \
- name=`expr x"$$base" : x'\(.*\)-'`; \
- action=`expr x"$$base" : x'.*-\(.*\)'`; \
- { \
- echo '#!/bin/sh'; \
- echo '# DO NOT EDIT! GENERATED AUTOMATICALLY!'; \
- echo; \
- echo '# Ensure proper definition of $$testsrcdir.'; \
- echo '. ./defs-static || exit 99'; \
- echo 'test -n "$$testsrcdir" || exit 99 # sanity check'; \
- echo; \
- echo "instspc_test_name='$$name'"; \
- echo "instspc_action='test-$$action'"; \
- echo ". \$$testsrcdir/instspc-tests.sh"; \
- } > $@-t
- $(AM_V_at)chmod a+rx $@-t && mv -f $@-t $@
-
-# All instspc-*.test tests work by sourcing the `instspc-tests.sh'
-# script. Also, they all use shared data generated by the helper
-# test `instspc-data.test', for reasons of speed.
-instspc-data.log: instspc-tests.sh
-$(instspc_tests:.test=.log): instspc-tests.sh instspc-data.log
+ && $(SHELL) ./gen-wrap-tests) > wrap-tests.tmp
+ $(AM_V_at)chmod a-w wrap-tests.tmp && mv -f wrap-tests.tmp $@
+
+# These tests are not expected to exist on filesystem, so we need this
+# declaration to avoid spurious failures.
+$(wrap_TESTS):
+# All the "wrapper" tests work by sourcing the script `wrap-tests.sh'.
+$(wrap_LOGS): wrap-tests.sh
+
+# Extra dependencies for hand-written tests.
+
+test-driver-custom-multitest.log: trivial-test-driver
+test-driver-custom-multitest-recheck.log: trivial-test-driver
+test-driver-custom-multitest-recheck2.log: trivial-test-driver
+test-driver-custom-html.log: trivial-test-driver
+
+testsuite-summary-color.log testsuite-summary-count.log: \
+ testsuite-summary-checks.sh extract-testsuite-summary
+
+testsuite-summary-count-many.log: trivial-test-driver
+testsuite-summary-count-many.log: extract-testsuite-summary
+
+$(tap_with_common_setup_TESTS:.test=.log): tap-common-setup.log tap-setup.sh
+
+tap-summary.log tap-summary-color.log: tap-summary-aux.sh
+
+# If two test scripts have the same basename, they will end up sharing
+# the same log file, leading to all sort of undefined and undesired
+# behaviours.
+check-no-repeated-test-name:
+ @LC_ALL=C; export LC_ALL; \
+ lst='$(TEST_LOGS)'; for log in $$lst; do echo $$log; done \
+ | sort | uniq -c | awk '($$1 > 1) { print }' \
+ | sed 's/\.log$$//' | grep . >&2 \
+ && { \
+ echo $@: test names listed above are duplicated >&2; \
+ exit 1; \
+ }; :
+check-local: check-no-repeated-test-name
+.PHONY: check-no-repeated-test-name
+
+.PHONY: maintainer-check-list-of-tests
+maintainer-check-list-of-tests:
+ @if diff -u /dev/null /dev/null >/dev/null 2>&1; then \
+ diff='diff -u'; \
+ else \
+ diff='diff'; \
+ fi; \
+ set X $(checked_test_extensions); shift; \
+ if test $$# -eq 0; then \
+ set X $(TEST_EXTENSIONS); shift; \
+ fi; \
+ if test $$# -eq 0; then \
+ echo '$@: internal error: $$(checked_test_extensions) and' \
+ '$$(TEST_EXTENSIONS) are both undefined or empty' >&2; \
+ exit 255; \
+ fi; \
+ lst='$(expected_list_of_tests)'; \
+ test -n "$$lst" || lst='$(TESTS)'; \
+ for t in $$lst; do \
+ echo "$$t"; \
+ done | sort >$(am__tmk); \
+ for ext in $$*; do \
+ ls *$$ext 2>/dev/null; \
+ if test $(srcdir) != $(builddir); then \
+ (cd $(srcdir) && ls *$$ext 2>/dev/null); \
+ fi; \
+ done | sort | uniq >$(am__tfs); \
+ if $$diff $(am__tmk) $(am__tfs) >$(am__tdf); then \
+ result=0; \
+ else \
+ echo '$@: list of tests in Makefile an on filesystem differ' >&2; \
+ echo "+ $$diff in-makefile on-filesystem" >&2; \
+ cat $(am__tdf) >&2; \
+ result=1; \
+ fi; \
+ rm -f $(am__tmk) $(am__tfs) $(am__tdf); \
+ exit $$result;
+
+.PHONY: clean-maintcheck-testslist-tmp
+clean-local: clean-maintcheck-testslist-tmp
+clean-maintcheck-testslist-tmp:
+ rm -f $(am__tmk) $(am__tfs) $(am__tdf)
# Dependencies valid for each test case.
$(TEST_LOGS): defs defs-static aclocal-$(APIVERSION) automake-$(APIVERSION)
+# FIXME: this should be made more granular once we have a cleaner
+# subdivision of the tests.
+$(TEST_LOGS): plain-functions.sh tap-functions.sh
clean-local: clean-local-check
.PHONY: clean-local-check
To run only selected tests:
- make -k check TESTS="foo.test bar.test"
+ make -k check TESTS="foo.test bar.test" (GNU make)
+ env TESTS="foo.test bar.test" make -e -k check (non-GNU make)
- For non-GNU make, you might have to use this instead:
+ To run the tests in cross-compilation mode:
- env TESTS="foo.test bar.test" make -e -k check
+ make -k check host_alias="$host_alias" (GNU make)
+ env host_alias="$host_alias" make -e -k check (non-GNU make)
+
+ Here `$host_alias' should be defined to a proper value different from
+ configure-determined `$build_alias', and should refer to a set of
+ cross-compilers you have available on your system; for example, if
+ on Linux you have a set of MinGW-targeted cross-compilers named
+ 'i586-mingw32msvc-cc', 'i586-mingw32msvc-c++', etc., you could use:
+
+ make -k check host_alias='i586-mingw32msvc'
Interpretation
Other:
SKIP - skipped tests (third party tools not available)
+ ERROR - some unexpected error condition
Getting details from failures
-----------------------------
- Each test is a shell script, and by default is run by /bin/sh.
- In a non-VPATH build you can run them directly, they will be verbose.
- By default, verbose output of a test foo.test is retained in the log
- file foo.log. A summary log is created in the file test-suite.log.
+ By default, verbose output of a test `foo.test' or `foo.tap' is
+ retained in the log file `foo.log'. A summary log is created in
+ the file `test-suite.log'.
You can limit the set of files using the TESTS variable, and enable
detailed test output at the end of the test run with the VERBOSE
env VERBOSE=x TESTS='first.test second.test ...' make -e check
+ You can also run the tests by hand, as explained in the next subsection.
+
+
+About the tests
+---------------
+
+ There are two kinds of tests in the Automake testsuite (both implemented
+ as shell scripts). The scripts with the `.test' suffix are "simple"
+ tests, their outcome completely determined by their exit status. Those
+ with the `.tap' suffix use the TAP protocol. If you want to run a test
+ by hand, you can do so directly if it is a simple test:
+
+ ./nogzip.test
+
+ (it will be verbose by default), while if it is a TAP test you can pass
+ it to your preferred TAP runner, as in e.g.:
+
+ prove --verbose --merge ./add-missing.tap
+
+ The tests can also be run directly in a VPATH build, as with:
+
+ /path/to/srcdir/tests/nogzip.test
+ prove --verbose --merge /path/to/srcdir/tests/add-missing.tap
+
Supported shells
----------------
+ By default, the tests are run by the $SHELL detected at configure
+ time. They also take care to re-execute themselves with that shell,
+ unless told not to. So, to run the tests with a different shell, say
+ `/path/to/another/sh', the user must use:
+
+ AM_TESTS_REEXEC=no /path/to/another/sh ./foo.test
+ AM_TESTS_REEXEC=no prove -v -e /path/to/another/sh ./bar.tap
+
+ to run a test directly, and:
+
+ make check LOG_COMPILER=/path/to/sh (GNU make)
+ LOG_COMPILER=/path/to/sh make -e check (non-GNU make)
+
+ to run the test(s) through the makefile test driver.
+
The test scripts are written with portability in mind, so that they
should run with any decent Bourne-compatible shell.
version of Zsh is used. Thus, if you want to run a test script, say
foo.test, with Zsh 4.2, you *can't* simply do `zsh foo.test', but
you *must* resort to:
- zsh -o no_function_argzero foo.test
+ AM_TESTS_REEXEC=no zsh -o no_function_argzero foo.test
Note that this problem does not occur if Zsh is executed through
a symlink with a basename of `sh', since in that case Zsh starts
instead.
For tests that use the `parallel-tests' Automake option, set the shell
- variable `parallel_tests' to "yes" before including ./defs. Also,
- do not use for them a name that ends in `-p.test', since that would
- risk to clash with automatically-generated tests. For tests that are
- *not* meant to work with the `parallel-tests' Automake option (these
- should be very very few), set the shell variable `parallel_tests' to
- "no" before including ./defs.
-
- ./defs sets a skeleton configure.in. If possible, append to this
- file. In some cases you'll have to overwrite it, but this should
- be the exception. Note that configure.in registers Makefile.in
- but do not output anything by default. If you need ./configure
- to create Makefile, append AC_OUTPUT to configure.in.
+ variable `parallel_tests' to "yes" before including ./defs. For tests
+ that are *not* meant to work with the `parallel-tests' Automake option
+ (these should be very very few), set the shell variable `parallel_tests'
+ to "no" before including ./defs.
+
+ Some tests in the Automake testsuite are auto-generated; those tests
+ might have custom extensions, but their basename (that is, with such
+ extension stripped) is expected to end with the "-w" string. For
+ example, the name of an auto-generated test can be `color-w.ptest'
+ or `ar-lib-w.shtst'. Please don't name hand-written tests in a way
+ that could cause them to be confused with auto-generated tests; for
+ example, `u-v-w.test' or `option-w.tap' are *not* valid name for
+ hand-written tests.
+
+ ./defs brings in some commonly required files, and sets a skeleton
+ configure.in. If possible, append to this file. In some cases
+ you'll have to overwrite it, but this should be the exception. Note
+ that configure.in registers Makefile.in but do not output anything by
+ default. If you need ./configure to create Makefile, append AC_OUTPUT
+ to configure.in. In case you don't want ./defs to pre-populate your
+ test directory (which is a rare occurrence), set `$am_create_destdir'
+ to "empty" before sourcing ./defs.
By default, the testcases are run with the `errexit' shell flag on,
to make it easier to catch failures you might not have thought of.
--- /dev/null
+#!/bin/sh
+# Copyright (C) 1996, 2000, 2001, 2002, 2010, 2011 Free Software
+# Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Test for various older bugs related to quoting, escaping and
+# line breaking in the use of AC_OUTPUT. Synthesised by a bunch
+# of older tests (referenced below).
+
+. ./defs || Exit 1
+
+plan_ 22
+
+rm -f configure.in depcomp # Not required.
+
+# -----------------------------------------------------------------------
+
+# Test for bug reported by François Pinard.
+# If \ is in AC_OUTPUT, automake barfs.
+# This was the old test `acoutbs.test'.
+
+# Also test for bug reported by David A. Swierczek.
+# Another bug with \ in AC_OUTPUT (this time with whitespace).
+# This was the old test `acoutbs2.test'.
+
+acoutbs_check ()
+{
+ dir=acoutbs$1
+ mkdir $dir
+ cd $dir
+ cat > configure.in
+ touch Makefile.am zot.in
+ command_ok_ "aclocal groks '\\' in AC_OUTPUT ($dir)" $ACLOCAL
+ command_ok_ "automake groks '\\' in AC_OUTPUT ($dir)" $AUTOMAKE
+ command_ok_ "autoconf groks '\\' in AC_OUTPUT ($dir)" $AUTOCONF
+ command_ok_ "can ./configure in $dir" ./configure
+ command_ok_ "zot created in $dir" test -f zot
+ ls -a > lst || bailout_ "cannot get file listing in $dir"
+ command_ok_ "'\\' not leaked in filenames in $dir" not grep '\\' lst
+ cd ..
+}
+
+acoutbs_check 1 << 'END'
+AC_INIT
+AM_INIT_AUTOMAKE(nonesuch, nonesuch)
+AC_OUTPUT(Makefile \
+zot)
+END
+
+acoutbs_check 2 << 'END'
+AC_INIT
+AM_INIT_AUTOMAKE(nonesuch, nonesuch)
+AC_ARG_PROGRAM
+AC_PROG_INSTALL
+AC_PROG_MAKE_SET
+AC_OUTPUT(\
+ Makefile \
+ zot
+)
+END
+
+# -----------------------------------------------------------------------
+
+# Test for bug reported by Jerome Santini.
+# If I put this line in my configure.in:
+# AC_OUTPUT(Makefile src/Makefile tests/Makefile, echo timestamp > stamp-h)dnl
+# automake is not happy:
+# [ ... ]
+# This was the old test `acoutnoq.test'.
+
+mkdir acoutnoq
+cd acoutnoq
+
+cat > configure.in << 'END'
+AC_INIT
+AM_INIT_AUTOMAKE(nonesuch, nonesuch)
+AC_OUTPUT(Makefile, [true])
+END
+
+: > Makefile.am
+
+command_ok_ "aclocal and quoted AC_OUTPUT second argument" $ACLOCAL
+command_ok_ "automake and quoted AC_OUTPUT second argument" $AUTOMAKE
+
+cd ..
+
+# -----------------------------------------------------------------------
+
+# Test for bug when AC_OUTPUT has 2 args on the same line, eg:
+# AC_OUTPUT([Makefile automake tests/Makefile],[chmod +x automake])
+# This was the old test `acoutpt.test'.
+
+mkdir acoutpt
+cd acoutpt
+
+cat > configure.in << 'END'
+AC_INIT
+AM_INIT_AUTOMAKE(nonesuch, nonesuch)
+AC_OUTPUT([Makefile], [true])
+END
+
+: > Makefile.am
+
+command_ok_ "aclocal and two AC_OUTPUT arguments on same line" $ACLOCAL
+command_ok_ "automake and two AC_OUTPUT arguments on same line" $AUTOMAKE
+
+cd ..
+
+# -----------------------------------------------------------------------
+
+# Test for bug reported by Eric Magnien.
+# This was the old test `acoutpt2.test'.
+
+mkdir acoutpt2
+cd acoutpt2
+
+# Name of the current "subtest".
+cur=acoutput2
+
+cat > configure.in <<END
+AC_INIT([$cur], [1.0])
+AM_INIT_AUTOMAKE
+AC_OUTPUT([subdir/Makefile subdir/foo Makefile foo], [true])
+END
+
+mkdir subdir
+
+: > Makefile.am
+: > subdir/Makefile.am
+: > foo.in
+: > subdir/foo.in
+
+command_ok_ "aclocal and AC_OUTPUT ($cur)" $ACLOCAL
+command_ok_ "automake and AC_OUTPUT ($cur)" $AUTOMAKE
+command_ok_ "foo.in mentioned two times in Makefile.in ($cur)" \
+ test `$FGREP foo.in Makefile.in | wc -l` -eq 2
+
+# This ought to work as well.
+command_ok_ "'automake -a -f' and AC_OUTPUT ($cur)" \
+ $AUTOMAKE --add-missing --force-missing
+
+cd ..
+
+# -----------------------------------------------------------------------
+
+# Test for bug reported by François Pinard.
+# This was the old test `acoutqnl.test'.
+
+mkdir acoutqnl
+cd acoutqnl
+
+cat > configure.in << 'END'
+AC_INIT
+AM_INIT_AUTOMAKE(nonesuch, nonesuch)
+AC_OUTPUT([Makefile],
+[echo zardoz has spoken])
+END
+
+: > Makefile.am
+
+command_ok_ "aclocal and two AC_OUTPUT arguments on two lines" $ACLOCAL
+command_ok_ "automake and two AC_OUTPUT arguments on two lines" $AUTOMAKE
+
+cd ..
+
+# -----------------------------------------------------------------------
+
+# And we're done.
+Exit 0
# Make sure m4_included files are also scanned for definitions.
# Report from Phil Edwards.
-required=GNUmake
. ./defs || Exit 1
cat >> configure.in << 'END'
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Make sure the "deleted header file" issue is fixed wrt. aclocal.m4
-# dependencies.
+# Make sure the "deleted header file" issue is fixed w.r.t. aclocal.m4
+# dependencies. See also related the tests `remake-deleted-m4-file.test'
+# and 'remake-renamed-m4-macro-and-file.test'.
. ./defs || Exit 1
cat >Makefile.am <<EOF
ACLOCAL_AMFLAGS = -I .
EOF
+
$ACLOCAL -I .
$AUTOMAKE
$AUTOCONF
+
./configure
$MAKE
grep GREPFOO Makefile
grep GREPBAR Makefile && Exit 1
+
sed 's/FOO/BAR/' < configure.in > t
mv -f t configure.in
rm -f foo.m4
$MAKE
grep GREPFOO Makefile && Exit 1
grep GREPBAR Makefile
+
+:
# Make sure dependencies on aclocal.m4 are set correctly.
# Report from Jim Meyering.
-# We need GNU make for the `rebuild rules' to be triggered.
-required=GNUmake
+required=cc
. ./defs || Exit 1
cat >>configure.in <<EOF
$MAKE check-not-foo
# Update one of the macros. This should cause ./configure, Makefile.in,
-# Makefile, lib/Makefile.in, and lib/Makfile to be updated. This assumes
-# GNU make.
+# Makefile, lib/Makefile.in, and lib/Makefile to be updated.
cat >m4/mymacro.m4 <<'EOF'
AC_DEFUN([MY_MACRO], [AC_LIBOBJ([foo])])
EOF
+using_gmake || $MAKE
$MAKE check-foo
+
+:
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Test to make sure that aclocal.m4's dependencies are honored
-# in sub-directories.
+# Test to make sure that aclocal.m4's dependencies are honored in
+# sub-directories. See also related tests `remake-subdir*.test'.
-# This test assumes that the `make' utility is able to start
-# over and reload Makefiles which have been remade (a non-POSIX feature).
-required='GNUmake'
. ./defs || Exit 1
cat >> configure.in << 'END'
# Make sure aclocal.m4 is rebuilt whenever a configure
# dependency changes. Test for acinclude.m4 and VPATH too.
-required='GNUmake'
. ./defs || Exit 1
cat >> configure.in << 'END'
+++ /dev/null
-#!/bin/sh
-# Copyright (C) 1996, 2000, 2001, 2002, 2010, 2011 Free Software
-# Foundation, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2, or (at your option)
-# any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program. If not, see <http://www.gnu.org/licenses/>.
-
-# Test for bug reported by Jerome Santini.
-# If I put this line in my configure.in:
-# AC_OUTPUT(Makefile src/Makefile tests/Makefile, echo timestamp > stamp-h)dnl
-# automake is not happy:
-# [ ... ]
-
-. ./defs || Exit 1
-
-cat > configure.in << 'END'
-AC_INIT
-AM_INIT_AUTOMAKE(nonesuch, nonesuch)
-AC_OUTPUT(Makefile, [true])
-END
-
-: > Makefile.am
-
-$ACLOCAL
-$AUTOMAKE
+++ /dev/null
-#!/bin/sh
-# Copyright (C) 1996, 2000, 2001, 2002, 2010, 2011 Free Software
-# Foundation, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2, or (at your option)
-# any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program. If not, see <http://www.gnu.org/licenses/>.
-
-# Test for bug when AC_OUTPUT has 2 args on the same line, eg:
-# AC_OUTPUT([Makefile automake tests/Makefile],[chmod +x automake])
-
-. ./defs || Exit 1
-
-cat > configure.in << 'END'
-AC_INIT
-AM_INIT_AUTOMAKE(nonesuch, nonesuch)
-AC_OUTPUT([Makefile], [true])
-END
-
-: > Makefile.am
-
-$ACLOCAL
-$AUTOMAKE
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Test that automake complains when required auxiliary files are not
+# found, and that `automake --add-missing' installs the files (and only
+# the files) it's supposed to, and that these files are symlinked by
+# default, but copied if the `--copy' option is used.
+
+am_create_testdir=empty
+. ./defs || Exit 1
+
+plan_ "later"
+
+build_aux=build-aux
+ocwd=`pwd` || fatal_ "cannot get current working directory"
+
+# Try to improve readability of displayed diffs.
+if diff -u /dev/null /dev/null; then
+ am_diff='diff -u'
+elif diff -c /dev/null /dev/null; then
+ am_diff='diff -c'
+else
+ am_diff=diff
+fi
+
+cat > configure.stub << END
+AC_INIT([$me], [1.0])
+AC_CONFIG_AUX_DIR([$build_aux])
+AM_INIT_AUTOMAKE
+AC_CONFIG_FILES([Makefile])
+END
+
+# Pre-compute aclocal.m4, in order to save several aclocal invocations.
+cat configure.stub - > configure.in <<'END'
+AC_PROG_CC
+AC_CANONICAL_BUILD
+AC_CANONICAL_HOST
+AC_CANONICAL_TARGET
+AC_CANONICAL_SYSTEM
+AM_PATH_LISPDIR
+AM_PATH_PYTHON
+END
+$ACLOCAL || framework_failure_ "cannot pre-compute aclocal.m4"
+
+rm -rf configure.in autom4te*.cache
+mv aclocal.m4 aclocal.stub
+
+# For debugging.
+cat configure.stub
+cat aclocal.stub
+
+check_count=0
+
+# This is hacky and ugly and complex, but allow us to organize our tests
+# below in a more "declarative fashion". All in all, a good trade-off.
+check_ ()
+{
+ set +x # Temporary disable shell traces to remove noise from log files.
+ incr_ check_count
+ echo check count: $check_count
+ override=no
+ run_aclocal=no
+ fetch_file=
+ while test $# -gt 0; do
+ case $1 in
+ --override) override=yes;;
+ --run-aclocal) run_aclocal=yes;;
+ --fetch-file) fetch_file=$2; shift;;
+ *) framework_failure_ "check_: invalid argument '$1'";;
+ esac
+ shift
+ done
+ mkdir testdir-$check_count
+ cd testdir-$check_count
+ # Directory for common data files (specific to the current test, but
+ # shared by its "subtests").
+ mkdir generic
+ cd generic
+ # Use `echo > ...', not `: > ...', here and below, as Solaris 10 /bin/sh
+ # might not execute the latter (the Autoconf manual gives more details).
+ echo > Makefile.am
+ if test $override = yes; then
+ echo > configure.in
+ else
+ cp "$ocwd"/configure.stub configure.in
+ fi
+ test -z "$fetch_file" || cp "$ocwd/$fetch_file" .
+ # Read description of "test scenario" from standard input.
+ what= line= name= files=
+ while read line; do
+ case $line in
+ '== Name ==') what=NAME;;
+ '== Makefile.am ==') what=Makefile.am;;
+ '== configure.in ==') what=configure.in;;
+ '== Files ==') what=LIST;;
+ '==*') framework_failure_ "invalid input line: $line";;
+ ''|'#%'*) : Empty line or ad-hoc comment, ignore. ;;
+ *)
+ if test $what = LIST; then
+ files="$files $line"
+ elif test $what = NAME; then
+ name=$line
+ else
+ printf '%s\n' "$line" >> "$what"
+ fi
+ ;;
+ esac
+ done
+ test -n "$name" || fatal_ "name of a test case not specified"
+ if test $run_aclocal = yes; then
+ if $ACLOCAL; then
+ ok_ "[$name] aclocal.m4 rebuilt"
+ echo == aclocal.m4 ==
+ cat aclocal.m4
+ else
+ not_ok_ "[$name] aclocal failure, aclocal.m4 not rebuilt"
+ fi
+ else
+ cp "$ocwd"/aclocal.stub aclocal.m4
+ fi
+ echo == Makefile.am ==
+ cat Makefile.am
+ echo == configure.in ==
+ cat configure.in
+ echo Expected files: $files
+ mkdir "$build_aux"
+ cd ..
+ # End of "test scenario" setup.
+ set -x # Re-enable shell traces.
+ pwd
+ ls -l generic
+ # Test once with `--copy', once without.
+ for action in link copy; do
+ case $action in
+ link) opts='--add-missing';;
+ copy) opts='-a --copy';;
+ *) fatal_ "invalid value '$action' for \$action";;
+ esac
+ pfx="[$action $name]"
+ cp -R generic $action
+ cd $action
+ # If the required auxiliary files are missing, and automake is
+ # not told to install them, it should complain and error out,
+ # and also give a useful suggestion.
+ AUTOMAKE_fails -d "$pfx missing files, automake fails"
+ for f in $files; do
+ command_ok_ \
+ "$pfx warn about missing file $f" \
+ grep "required file ['\`]$build_aux/$f' not found" stderr
+ # Suggest the user to use `--add-missing'.
+ command_ok_ \
+ "$pfx suggest --add-missing for $f" \
+ grep ".*--add-missing.* install .*$f" stderr
+ done
+ # No files should be automatically installed by automake if it
+ # is not told to.
+ if ls "$build_aux" | grep .; then r='not ok'; else r=ok; fi
+ result_ "$r" "$pfx no extra files installed"
+ AUTOMAKE_run -d "$pfx automake run successfully" -- $opts
+ ls -l . $build_aux
+ # The expected files should get installed correctly (e.g., no
+ # broken symlinks).
+ for f in $files; do
+ command_ok_ \
+ "$pfx file $f installed" \
+ test -f $build_aux/$f
+ done
+ # Automake should inform about which files it's installing.
+ for f in $files; do
+ command_ok_ \
+ "$pfx report installation of $f" \
+ grep ": installing ['\`]$build_aux/$f'$" stderr
+ done
+ # Only the expected files should be installed. But automake always
+ # require `missing' and `install-sh', so account for them.
+ all_files="install-sh missing $files"
+ for f in $all_files; do echo $f; done | sort | uniq > files.exp
+ (cd $build_aux && ls) | sort > files.got
+ cat files.exp
+ cat files.got
+ command_ok_ \
+ "$pfx all and only expected files installed" \
+ $am_diff files.exp files.got
+ # The files should be copied by `--copy' and symlinked otherwise.
+ for f in $files; do
+ if test -h $build_aux/$f; then
+ is_symlink=yes
+ else
+ is_symlink=no
+ fi
+ case $action,$is_symlink in
+ link,yes) ok_ "$pfx $f has been symlinked" ;;
+ link,no) not_ok_ "$pfx $f has not been symlinked" ;;
+ copy,yes) not_ok_ "$pfx $f has been symlinked" ;;
+ copy,no) ok_ "$pfx $f has not been symlinked" ;;
+ *) fatal_ "invalid condition in case" ;;
+ esac
+ done
+ # Now that the required auxiliary files have been installed, automake
+ # should not complain anymore even if the `--add-missing' option is
+ # not used.
+ AUTOMAKE_run -d "$pfx automake finds all added files"
+ cd ..
+ done # for action in link copy
+ cd "$ocwd" || fatal_ "cannot chdir back to top-level test directory"
+}
+
+# Automake should always and unconditionally require the `missing'
+# and `install-sh' scripts.
+check_ <<'END'
+== Name ==
+minimal
+== Files ==
+install-sh missing
+END
+
+check_ <<'END'
+== Name ==
+depcomp/C
+== Files ==
+depcomp
+== configure.in ==
+AC_PROG_CC
+== Makefile.am ==
+bin_PROGRAMS = foo
+END
+
+check_ <<'END'
+== Name ==
+depcomp/C++
+== Files ==
+depcomp
+== configure.in ==
+AC_PROG_CXX
+== Makefile.am ==
+bin_PROGRAMS = foo
+foo_SOURCES = foo.cc
+END
+
+check_ --run-aclocal <<'END'
+== Name ==
+compile
+== Files ==
+compile
+== configure.in ==
+# Using AM_PROG_CC_C_O in configure.in should be enough. No need to
+# use AC_PROG_CC too, nor to define xxx_PROGRAMS in Makefile.am.
+AM_PROG_CC_C_O
+END
+
+# For config.guess and config.sub.
+for mach in build host target system; do
+ MACH=`echo "$mach" | LC_ALL=C tr '[a-z]' '[A-Z]'`
+ check_ <<END
+== Name ==
+cfg-$mach
+== Files ==
+config.sub
+config.guess
+== configure.in ==
+AC_CANONICAL_$MACH
+END
+done
+
+check_ <<'END'
+== Name ==
+ylwrap/Lex
+== Files ==
+ylwrap
+== configure.in ==
+AC_PROG_CC
+AC_PROG_LEX
+== Makefile.am ==
+AUTOMAKE_OPTIONS = no-dependencies
+bin_PROGRAMS = foo
+foo_SOURCES = foo.l
+END
+
+check_ <<'END'
+== Name ==
+ylwrap/Yacc
+== Files ==
+ylwrap
+== configure.in ==
+AC_PROG_CC
+AC_PROG_YACC
+== Makefile.am ==
+AUTOMAKE_OPTIONS = no-dependencies
+bin_PROGRAMS = foo
+foo_SOURCES = foo.y
+END
+
+echo '@setfilename foo.info' > foo.texi
+check_ --fetch-file foo.texi <<'END'
+== Name ==
+Texinfo
+== Files ==
+texinfo.tex
+== Makefile.am ==
+info_TEXINFOS = foo.texi
+END
+
+echo '@include version.texi' >> foo.texi
+check_ --fetch-file foo.texi <<'END'
+== Name ==
+Texinfo/mdate-sh
+== Files ==
+mdate-sh
+texinfo.tex
+== Makefile.am ==
+info_TEXINFOS = foo.texi
+END
+
+rm -f foo.texi
+
+check_ <<'END'
+== Name ==
+elisp-comp
+== Files ==
+elisp-comp
+== configure.in ==
+AM_PATH_LISPDIR
+== Makefile.am ==
+lisp_LISP = foo.el
+END
+
+check_ <<'END'
+== Name ==
+py-compile
+== Files ==
+py-compile
+== configure.in ==
+AM_PATH_PYTHON
+== Makefile.am ==
+python_PYTHON = foo.py
+END
+
+# Try few unrelated auxiliary scripts together.
+check_ <<'END'
+== Name ==
+misc
+== Files ==
+py-compile
+depcomp
+ylwrap
+config.sub
+config.guess
+== configure.in ==
+AC_CANONICAL_BUILD
+AC_CANONICAL_HOST
+AC_PROG_CXX
+== Makefile.am ==
+PYTHON = python
+pythondir = $(prefix)/py
+YACC = bison -y
+bin_PROGRAMS = foo
+foo_SOURCES = bar.yxx baz.c++
+python_PYTHON = zardoz.py
+END
+
+:
targets='all install-exec install-data uninstall'
echo "$targets:" | sed -e 's/[ :]/-local&/g' > Makefile.am
-cat Makefile.am # might be useful for debugging
+cat Makefile.am # For debugging.
$ACLOCAL
$AUTOMAKE
for target in $targets; do
- grep "${target}-local" Makefile.in # might be useful for debugging
+ grep "${target}-local" Makefile.in # For debugging.
grep "${target}-am:.*${target}-local" Makefile.in
done
$AUTOMAKE
./configure
-# make distdir should fail because NEWS does not mention 1.0a
+# "make distdir" should fail because NEWS does not mention 1.0a
$MAKE check 2>stderr && { cat stderr >&2; Exit 1; }
cat stderr >&2
grep 'NEWS not updated' stderr
parallel_tests=yes
. ./defs || Exit 1
-set -e
-
cat >> configure.in << 'END'
AC_PROG_MKDIR_P
AC_OUTPUT
TEST_EXTENSIONS = .sh .test
TESTS = foo.test bar.sh
SH_LOG_COMPILER = sh
-AM_TESTS_ENVIRONMENT = $(MKDIR_P) quux.dir; exec 9>&2; fd=9; export fd;
-EXTRA_DIST = $(TESTS)
+AM_TESTS_ENVIRONMENT = \
+ $(MKDIR_P) quux.dir; \
+ if test -f $(srcdir)/test-env.sh; then \
+ . $(srcdir)/test-env.sh; \
+ fi; \
+ FOO=1; export FOO;
+EXTRA_DIST = $(TESTS) test-env.sh
END
cat > foo.test << 'END'
#! /bin/sh
-echo "FOO='$FOO'"
-echo "BAR='$BAR'"
-test x"$FOO" = x"$BAR"
+ls -l && test -d quux.dir
END
chmod a+x foo.test
cat > bar.sh << 'END'
-: ${fd=2}
-eval "echo 'diagnostic message from test bar' >&${fd}"
+echo "FOO='$FOO'"
+echo "BAR='$BAR'"
+test x"$FOO" = x"$BAR"
END
-minicheck ()
+debug_info ()
{
+ cat test-suite.log
cat foo.log
cat bar.log
+}
+
+minicheck ()
+{
+ debug_info
test -d quux.dir
}
./configure
-FOO=1; export FOO
-
-TESTS_ENVIRONMENT='BAR=1' $MAKE check || { cat test-suite.log; Exit 1; }
+TESTS_ENVIRONMENT='BAR=1' $MAKE check || { debug_info; Exit 1; }
minicheck
miniclean
-TESTS_ENVIRONMENT='BAR=2' $MAKE check && { cat test-suite.log; Exit 1; }
-minicheck
-miniclean
-
-BAR=$FOO; export BAR
-$MAKE check 2>stderr || { cat stderr >&2; Exit1; }
-cat stderr >&2
+TESTS_ENVIRONMENT='BAR=2' $MAKE check && { debug_info; Exit 1; }
minicheck
-grep 'diagnostic message from test bar' stderr
-grep 'diagnostic message from test bar' bar.log && Exit 1
miniclean
-TESTS_ENVIRONMENT='fd=2' $MAKE check 2>stderr || { cat stderr >&2; Exit1; }
-cat stderr >&2
+echo 'BAR=1 && export BAR' > test-env.sh
+$MAKE check || { debug_info; Exit 1; }
minicheck
-grep 'diagnostic message from test bar' stderr && Exit 1
-grep 'diagnostic message from test bar' bar.log
-miniclean
-
$MAKE distcheck
:
. ./defs || Exit 1
-set -e
-
cp "$testsrcdir"/../doc/amhello-1.0.tar.gz . \
|| fatal_ "cannot get amhello tarball"
required=gcc
. ./defs || Exit 1
-set -e
-
cp "$testsrcdir"/../doc/amhello-1.0.tar.gz . \
|| fatal_ "cannot get amhello tarball"
required=i586-mingw32msvc-gcc
. ./defs || Exit 1
-set -e
-
cp "$testsrcdir"/../doc/amhello-1.0.tar.gz . \
|| fatal_ "cannot get amhello tarball"
foo2 += $(foo3)
foo3 = -Wno-error
bar = -Wportability
-## This will give a warning with `-Wportability'
+# This will give a warning with `-Wportability'.
zardoz :=
-## This would give a warning with `-Woverride'.
+# This would give a warning with `-Woverride'.
install:
END
$ACLOCAL
-AUTOMAKE_run 0
+AUTOMAKE_run
grep '^Makefile\.am:.*:=.*not portable' stderr
grep README stderr && Exit 1
$EGREP '(install|override)' stderr && Exit 1
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Make sure `ar-lib' wraps the Microsoft Library Manager (lib) correctly
+# Make sure `ar-lib' wraps the Microsoft Library Manager (lib) correctly.
-required=xsi-shell
+required=xsi-lib-shell
. ./defs || Exit 1
-cp "$top_testsrcdir/lib/ar-lib" .
+get_shell_script ar-lib
# Use a dummy lib, since lib isn't readily available on all systems.
cat >lib <<'END'
chmod +x ./lib
-# Check if ar-lib can create an archive with "cr"
+# Check if ar-lib can create an archive with "cr".
opts=`./ar-lib ./lib cr foo.lib foo.obj`
test x"$opts" = x"lib -NOLOGO -OUT:foo.lib foo.obj"
-# Check if ar-lib can update an existing archive with "r"
+# Check if ar-lib can update an existing archive with "r".
touch foo.lib
opts=`./ar-lib ./lib r foo.lib foo.obj`
test x"$opts" = x"lib -NOLOGO -OUT:foo.lib foo.lib foo.obj"
opts=`./ar-lib ./lib r foo.lib @list`
test x"$opts" = x"lib -NOLOGO -OUT:foo.lib foo.lib @list"
-# Check if ar-lib can delete a member from an archive with "d"
+# Check if ar-lib can delete a member from an archive with "d".
opts=`./ar-lib ./lib d foo.lib foo.obj`
test x"$opts" = x"lib -NOLOGO -REMOVE:foo.obj foo.lib"
-# Check if ar-lib can delete members in an @FILE
+# Check if ar-lib can delete members in an @FILE.
echo foo.obj > foolist
opts=`./ar-lib ./lib d foo.lib @foolist`
test x"$opts" = x"lib -NOLOGO -REMOVE:foo.obj foo.lib"
-# Check if ar-lib can list archive members with "t"
+# Check if ar-lib can list archive members with "t".
opts=`./ar-lib ./lib t foo.lib`
test x"$opts" = x"lib -NOLOGO -LIST foo.lib"
-# Check if ar-lib can extract archive members with "x"
+# Check if ar-lib can extract archive members with "x".
touch fake.lib
opts=`./ar-lib ./lib x fake.lib`
test x"$opts" = x"lib -NOLOGO -EXTRACT:fake.obj fake.lib"
-# Check if ar-lib can extract specified archive members with "x"
+# Check if ar-lib can extract specified archive members with "x".
opts=`./ar-lib ./lib x foo.lib foo.obj`
test x"$opts" = x"lib -NOLOGO -EXTRACT:foo.obj foo.lib"
-# Check if ar-lib can extract members in an @FILE
+# Check if ar-lib can extract members in an @FILE.
opts=`./ar-lib ./lib x foo.lib @foolist`
test x"$opts" = x"lib -NOLOGO -EXTRACT:foo.obj foo.lib"
-# Check if ar-lib passes -lib and -LTCG through to the wrappee
+# Check if ar-lib passes -lib and -LTCG through to the wrappee.
opts=`./ar-lib ./lib -lib -LTCG x foo.lib foo.obj`
test x"$opts" = x"lib -lib -LTCG -NOLOGO -EXTRACT:foo.obj foo.lib"
-# Check if ar-lib can extract backslashed members
+# Check if ar-lib can extract backslashed members.
touch fake2.lib
opts=`./ar-lib ./lib x fake2.lib`
test x"$opts" = x"lib -NOLOGO -EXTRACT:dir\\fake2.obj fake2.lib"
. ./defs || Exit 1
-set -e
-
re='Files .*automatically distributed.*if found'
# The automake manual states that the list of automatically-distributed
# Required when using Texinfo.
: > texinfo.tex
-cp "$testsrcdir/../lib/mdate-sh" .
+cp "$top_testsrcdir"/lib/mdate-sh .
$ACLOCAL
$AUTOCONF
## Now the checks.
@for f in $(autodist_list); do \
echo "file: $$f"; \
- ## Some filenames might contain dots, but this won't cause spurious
- ## failures, and "spurious successes" are so unlikely that they're
- ## not worth worrying about.
+## Some filenames might contain dots, but this won't cause spurious
+## failures, and "spurious successes" are so unlikely that they're
+## not worth worrying about.
echo ' ' $(DIST_COMMON) ' ' | grep "[ /]$$f " >/dev/null \
|| { echo $$f: distcom fail >&2; exit 1; }; \
done
# Related to automake bug#7819.
# Keep this test in sync with sister test `autodist-subdir.test'.
+am_create_testdir=empty
. ./defs || Exit 1
-# Ensure we are run from the right directory.
-# (The last thing we want is to delete some random user files.)
-test -f ../defs-static && test -f ../defs || Exit 99
-rm -f *
-
cat > configure.in <<END
AC_INIT([$me], [1.0])
AC_CONFIG_AUX_DIR([.])
echo "file: $$f"; \
test -f $(distdir)/$$f \
|| { echo $$f: distdir fail >&2; exit 1; }; \
- ## Some filenames might contain dots, but this won't cause spurious
- ## failures, and "spurious successes" are so unlikely that they're
- ## not worth worrying about.
+## Some filenames might contain dots, but this won't cause spurious
+## failures, and "spurious successes" are so unlikely that they're
+## not worth worrying about.
echo ' ' $(DIST_COMMON) ' ' | grep "[ /]$$f " >/dev/null \
|| { echo $$f: distcom fail >&2; exit 1; }; \
done
# Check rebuild rules for autoheader.
-required=GNUmake
. ./defs || Exit 1
cat >>configure.in <<EOF
AC_OUTPUT
EOF
-: > Makefile.am
: > foo.m4
+cat > Makefile.am <<'END'
+.PHONY: test
+check-local: test
+test: all
+ cat $(srcdir)/config.hin ;: For debugging.
+ cat config.h ;: Likewise.
+ grep '#.*GREPME' $(srcdir)/config.hin
+ grep '#.*define.*GREPME' config.h
+END
+
$ACLOCAL
$AUTOCONF
$AUTOHEADER
$sleep
echo 'AC_DEFINE([GREPME], 1, [Doc for GREPME])' > foo.m4
-$MAKE
-grep GREPME config.hin
-grep GREPME config.h
-
+$MAKE test
$MAKE distcheck
+
+:
# Check rebuild rules for AC_CONFIG_HEADERS.
# (This should also work without GNU Make.)
-required=gcc
+required=cc
. ./defs || Exit 1
cat >>configure.in <<'EOF'
: > sub1/config.top
echo '#define NAME "grepme1"' >sub2/config.bot
-echo SUBDIRS = sub3 >Makefile.am
-echo noinst_PROGRAMS = run >sub3/Makefile.am
+
+cat > Makefile.am <<'END'
+SUBDIRS = sub3
+.PHONY: test-prog-updated
+test-prog-updated:
+ stat older sub3/run$(EXEEXT) || : For debugging.
+ test `ls -t older sub3/run$(EXEEXT) | sed 1q` = sub3/run$(EXEEXT)
+END
+
+cat > sub3/Makefile.am <<'END'
+noinst_PROGRAMS = run
+END
cat >sub3/run.c <<'EOF'
#include <defs.h>
#include <config.h>
#include <stdio.h>
-int
-main ()
+int main (void)
{
puts (NAME); /* from config.h */
puts (PACKAGE); /* from defs.h */
$AUTOHEADER
$AUTOMAKE
-./configure
+# Do not reject slow dependency extractors: we need dependency tracking.
+./configure --enable-dependency-tracking
$MAKE
-sub3/run | grep grepme1
+# Sanity check.
+cross_compiling || { sub3/run | grep grepme1; }
+: > older
$sleep
echo '#define NAME "grepme2"' > sub2/config.bot
$MAKE
-sub3/run | grep grepme2
+cross_compiling || { sub3/run | grep grepme2; }
+$MAKE test-prog-updated
$MAKE distcheck
+
+:
. ./defs || Exit 1
cat >>configure.in <<'EOF'
-AC_PROG_CC
AC_CONFIG_HEADERS([config.h])
AC_OUTPUT
EOF
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2004, 2006, 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Test Automake's command-line options.
+
+. ./defs || Exit 1
+
+plan_ 17
+
+# Usage: bad_cmdline DESCRIPTION REGEX-FOR-STDERR [ARGS-FOR-AUTOMAKE...]
+do_check ()
+{
+ test $# -ge 3 || fatal_ "do_check: invalid usage"
+ desc=$1; shift
+ regex=$1; shift
+ AUTOMAKE_fails -d "$desc (run)" -- "$@"
+ command_ok_ "$desc (stderr)" grep "$regex" stderr
+}
+
+do_check 'invalid long option' 'unrecognized option.*--voo' --voo
+
+# Older perl has a buggy Getopt::Long which makes this fail.
+if $PERL -e 'require 5.8.2;'; then
+ do_check "list of options terminated by '--'" \
+ 'input file.*--voo' -- --voo
+else
+ skip_row_ 2 -r "older perl with buggy Getopt::Long"
+fi
+
+do_check "empty argument" \
+ 'empty argument' ''
+
+do_check "missing argument for long option" \
+ 'option.*-W.*requires an argument' -W
+
+do_check "missing argument for short option" \
+ 'option.*--warnings.*requires an argument' --warnings
+
+do_check "'--help' as option argument" \
+ 'unknown warning.*--help' --warnings --help
+
+do_check "'--help' as option argument" \
+ 'unknown warning.*--help' --warnings --help
+
+do_check "ambiguous incomplete option" \
+ 'unrecognized option.*--ver' --ver
+
+command_ok_ "unambiguous incomplete long option" $AUTOMAKE --vers
+
+:
+++ /dev/null
-#! /bin/sh
-# Copyright (C) 2004, 2006, 2011 Free Software Foundation, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2, or (at your option)
-# any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program. If not, see <http://www.gnu.org/licenses/>.
-
-# Test Automake's command-line options.
-. ./defs || Exit 1
-
-AUTOMAKE_fails --voo
-grep 'unrecognized option.*--voo' stderr
-# older perl has a buggy Getopt::Long which makes this fail.
-if $PERL -e 'require 5.8.2;'; then
- AUTOMAKE_fails -- --voo
- grep 'input file.*--voo' stderr
-fi
-AUTOMAKE_fails ''
-grep 'empty argument' stderr
-AUTOMAKE_fails -W
-grep 'option.*-W.*requires an argument' stderr
-AUTOMAKE_fails --warnings
-grep 'option.*--warning.*requires an argument' stderr
-AUTOMAKE_fails --warnings --help
-grep 'unknown warning.*--help' stderr
-AUTOMAKE_fails --ver
-grep 'unrecognized option.*--ver' stderr
-$AUTOMAKE --vers
-
-:
. ./defs || Exit 1
+plan_ 3
+
cat > configure.in <<END
AC_INIT([$me], [1.0])
AC_CONFIG_AUX_DIR([\$foo])
: > Makefile.am
-$ACLOCAL
+command_ok_ "aclocal" $ACLOCAL
# Both these two invocations are meant.
# They exercise both code paths concerning auxiliary files.
-$AUTOMAKE -a
-$AUTOMAKE
+command_ok_ "automake -a" -D TODO -r "long-standing limitation" $AUTOMAKE -a
+command_ok_ "automake" -D TODO -r "long-standing limitation" $AUTOMAKE
:
. ./defs || Exit 1
+plan_ 4
+
cat >configure.in <<END
AC_INIT([$me], [1.0])
AC_CONFIG_AUX_DIR([aux])
: > Makefile.am
-$ACLOCAL
-AUTOMAKE_fails
-grep '^configure\.in:2:.*aux.*W32' stderr
+am_warn_unportable_auxdir ()
+{
+ details=$1
+ AUTOMAKE_fails
+ command_ok_ \
+ "warn about $details unportable auxdir name" \
+ grep '^configure\.in:2:.*aux.*W32' stderr
+}
+
+$ACLOCAL || fatal_ "aclocal failed"
+
+am_warn_unportable_auxdir "non-existent"
if mkdir aux; then
- AUTOMAKE_fails
- grep '^configure\.in:2:.*aux.*W32' stderr
+ am_warn_unportable_auxdir "existent"
+else
+ skip_row_ 2 -r "cannot create directory named 'aux'"
fi
:
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Test usage of AM_INIT_AUTOMAKE with two ot three arguments, for
+# Test usage of AM_INIT_AUTOMAKE with two or three arguments, for
# backward-compatibility.
. ./defs || Exit 1
## Might be useful for debugging.
display:
- ## The following should be substituted by AM_INIT_AUTOMAKE.
+## The following should be substituted by AM_INIT_AUTOMAKE.
@echo PACKAGE = $(PACKAGE)
@echo VERSION = $(VERSION)
- ## The following should not be substituted, as we used the
- ## old form of AC_INIT.
+## The following should not be substituted, as we used the
+## old form of AC_INIT.
@echo PACKAGE_NAME = $(PACKAGE_NAME)
@echo PACKAGE_VERSION = $(PACKAGE_VERSION)
@echo PACKAGE_TARNAME = $(PACKAGE_TARNAME)
for ac_init in 'AC_INIT' 'AC_INIT([Makefile.am])'; do
for am_extra_args in '' ', []' ', [:]' ', [false]'; do
rm -rf autom4te*.cache config* Makefile.in Makefile
- cat > configure.in <<END
-$ac_init
-AM_INIT_AUTOMAKE([FooBar], [0.7.1]$am_extra_args)
-AC_CONFIG_FILES([Makefile])
-AC_OUTPUT
+ unindent > configure.in <<END
+ $ac_init
+ AM_INIT_AUTOMAKE([FooBar], [0.7.1]$am_extra_args)
+ AC_CONFIG_FILES([Makefile])
+ AC_OUTPUT
END
- cat configure.in # might be useful for debugging
+ cat configure.in # For debugging.
$ACLOCAL
$AUTOCONF
$AUTOMAKE
END
for am_arg3 in ':' 'false' '#' ' '; do
- cat > configure.in <<END
-AC_INIT
-AC_CONFIG_HEADERS([config.h])
-AM_INIT_AUTOMAKE([pkgname], [pkgversion], [$am_arg3])
-AC_OUTPUT
+ unindent > configure.in <<END
+ AC_INIT
+ AC_CONFIG_HEADERS([config.h])
+ AM_INIT_AUTOMAKE([pkgname], [pkgversion], [$am_arg3])
+ AC_OUTPUT
END
- cat configure.in # might be useful for debugging
+ cat configure.in # For debugging.
$AUTOCONF
./configure
- cat config.h # might be useful for debugging
+ cat config.h # For debugging.
# The non-empty third argument should prevent PACKAGE and VERSION
# from being AC_DEFINE'd.
$EGREP 'pkg(name|version)' config.h && Exit 1
+ # This is required because even relatively-recent versions of the
+ # BSD shell wrongly exit when the `errexit' shell flag is active if
+ # the last command of a compound statement fails, even if it should
+ # be protected by the use of "&&".
+ :
done
for am_extra_args in '' ',' ', []'; do
- cat > configure.in <<END
-AC_INIT
-AC_CONFIG_HEADERS([config.h])
-AM_INIT_AUTOMAKE([pkgname], [pkgversion]$am_extra_args)
-AC_OUTPUT
+ unindent > configure.in <<END
+ AC_INIT
+ AC_CONFIG_HEADERS([config.h])
+ AM_INIT_AUTOMAKE([pkgname], [pkgversion]$am_extra_args)
+ AC_OUTPUT
END
- cat configure.in # might be useful for debugging
+ cat configure.in # For debugging.
$AUTOCONF
./configure
- cat config.h # might be useful for debugging
+ cat config.h # For debugging.
grep '^ *# *define *PACKAGE *"pkgname" *$' config.h
grep '^ *# *define *VERSION *"pkgversion" *$' config.h
done
### Run 2 ###
cat > configure.in <<'END'
-dnl# `AC_INIT' in Autoconf <= 2.63 doesn't have an URL argument.
-dnl# Luckily, `AC_AUTOCONF_VERSION' and `m4_version_prereq' are
-dnl# both present in autoconf 2.62, which we require; so that we
-dnl# can at least use the following workaround.
+dnl: `AC_INIT' in Autoconf <= 2.63 doesn't have an URL argument.
+dnl: Luckily, `AC_AUTOCONF_VERSION' and `m4_version_prereq' are
+dnl: both present in autoconf 2.62, which we require; so that we
+dnl: can at least use the following workaround.
m4_version_prereq([2.64],
[AC_INIT([ac_name], [ac_version], [ac_bugreport], [ac_tarname],
[ac_url])],
: > foo3.in
: > sub/bar.in
-ls -l # might be useful for debugging
+ls -l # For debugging.
for args in \
'foo' \
'foo:foo1.in:foo2.in:sub/bar.in:foo3.in' \
; do
rm -rf autom4te*.cache aclocal.m4
- cat >configure.in <<END
-AC_INIT([$me], [1.0])
-AM_INIT_AUTOMAKE
-AC_CONFIG_FILES([$args])
-AC_OUTPUT
+ unindent >configure.in <<END
+ AC_INIT([$me], [1.0])
+ AM_INIT_AUTOMAKE
+ AC_CONFIG_FILES([$args])
+ AC_OUTPUT
END
- cat configure.in # for debugging
+ cat configure.in # For debugging.
$ACLOCAL
$AUTOMAKE Makefile
mv -f Makefile.in Makefile.acf
rm -rf autom4te*.cache aclocal.m4
- cat >configure.in <<END
-AC_INIT([$me], [1.0])
-AM_INIT_AUTOMAKE
-AC_OUTPUT([$args])
+ unindent >configure.in <<END
+ AC_INIT([$me], [1.0])
+ AM_INIT_AUTOMAKE
+ AC_OUTPUT([$args])
END
- cat configure.in # for debugging
+ cat configure.in # For debugging.
$ACLOCAL
$AUTOMAKE Makefile
mv -f Makefile.in Makefile.aco
# Backward-compatibility test: try to build and distribute a package
# using obsoleted forms of AC_INIT, AM_INIT_AUTOMAKE and AC_OUTPUT.
-# This script can also serve as mild stress-testing for Automke.
+# This script can also serve as mild stress-testing for Automake.
# See also the similar test `backcompat6.test'.
+parallel_tests=no
. ./defs || Exit 1
# Yuck!
cat > configure.in <<'END'
-dnl. Everything here is *deliberately* underquoted!
+dnl: Everything here is *deliberately* underquoted!
AC_INIT(src/foo.input)
AM_INIT_AUTOMAKE(foo, 1.0)
AC_CONFIG_FILES(Makefile:mkfile.in)
$AUTOMAKE
./configure
-ls -l . hacky src data tests # useful for debugging
+ls -l . hacky src data tests # For debugging.
test ! -f mkfile
$MAKE
$MAKE distdir
test -f $distdir.tar.gz
chmod a-x tests/a.test
-# dist-hook should take care of making test files executables
+# dist-hook should take care of making test files executables.
$MAKE distcheck
:
# Backward-compatibility test: try to build and distribute a package
# using obsoleted forms of AC_INIT, AM_INIT_AUTOMAKE and AC_OUTPUT.
-# This script can also serve as mild stress-testing for Automke.
+# This script can also serve as mild stress-testing for Automake.
# See also the similar test `backcompat5.test'.
+required=cc
. ./defs || Exit 1
# Anyone doing something like this in a real-life package probably
# deserves to be killed.
cat > configure.in <<'END'
-dnl. Everything here is *deliberately* underquoted!
+dnl: Everything here is *deliberately* underquoted!
AC_INIT(quux.c)
PACKAGE=nonesuch-zardoz
VERSION=nonesuch-0.1
AC_SUBST(three)
AC_PROG_CC
AC_CONFIG_HEADERS(config.h:config.hin)
+AM_CONDITIONAL(CROSS_COMPILING, test "$cross_compiling" = yes)
AC_OUTPUT(Makefile foo.got:foo1.in:foo2.in:foo3.in)
END
check-local:
test x'$(PACKAGE)' = x'nonesuch-zardoz'
test x'$(VERSION)' = x'nonesuch-0.1'
+if ! CROSS_COMPILING
./foo
- ## Do not anchor the regexps w.r.t. the end of line, because on
- ## MinGW/MSYS, grep may assume LF line endings only, while our
- ## `foo' program may generate CRLF line endings.
+## Do not anchor the regexps w.r.t. the end of line, because on
+## MinGW/MSYS, grep may assume LF line endings only, while our
+## `foo' program may generate CRLF line endings.
./foo | grep '^PACKAGE = nonesuch-zardoz!'
./foo | echo '^VERSION = nonesuch-0\.1!'
+endif
END
cat > quux.c <<'END'
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Make sure we diagnose trailing backslash at the end of a file.
-# Report from Akim Demaile <akim@epita.fr>
+# Report from Akim Demaile <akim@epita.fr>.
. ./defs || Exit 1
# Stress test on canonicalization.
-required='libtool libtoolize'
+required='cc libtool libtoolize'
. ./defs || Exit 1
cat >> configure.in << 'END'
AC_PROG_CC
AC_PROG_RANLIB dnl: for static libraries
AC_PROG_LIBTOOL dnl: for libtool libraries
+AM_CONDITIONAL([CROSS_COMPILING], [test "$cross_compiling" = yes])
AC_OUTPUT
END
-touch ,foo-bar libb.az+baz lib~zardoz,, || Exit 77
-rm -f ,foo-bar libb.az+baz lib~zardoz,,
+if touch ,foo-bar libb.az+baz lib~zardoz,,; then
+ rm -f ,foo-bar libb.az+baz lib~zardoz,,
+else
+ skip_ "cannot create regular files with \"tricky\" names"
+fi
cat > Makefile.am << 'END'
noinst_PROGRAMS = dummy_static dummy_dynamic ,foo-bar
check-local:
ls -l
+if CROSS_COMPILING
+ test -f ./,foo-bar$(EXEEXT)
+ test -f ./dummy_static$(EXEEXT)
+ test -f ./dummy_dynamic$(EXEEXT)
+else !CROSS_COMPILING
./,foo-bar
./dummy_static
./dummy_dynamic
./,foo-bar | grep 'Hello, FooBar!'
./dummy_static | grep 'Hello from Static!'
./dummy_dynamic | grep 'Hello from Dynamic!'
+endif !CROSS_COMPILING
END
cat > foobar.c << 'END'
required=gcc
. ./defs || Exit 1
-cat > configure.1 << 'END'
-AC_INIT(a.c)
-AM_INIT_AUTOMAKE(nonesuch, 0.23)
+cat >> configure.in << 'END'
AC_PROG_CC
AM_PROG_CC_C_O
-AC_OUTPUT(Makefile)
-END
-
-# This one makes sure that $CC can be used after AM_PROG_CC_C_O.
-cat > configure.3 << 'END'
-AC_INIT(a.c)
-AM_INIT_AUTOMAKE(nonesuch, 0.23)
-AC_PROG_CC
-AM_PROG_CC_C_O
-$CC -v > Hi 2>&1 || exit 1
-AC_OUTPUT(Makefile)
+# Make sure that $CC can be used after AM_PROG_CC_C_O.
+$CC -v || exit 1
+AC_OUTPUT
END
cat > Makefile.am << 'END'
}
END
-cat > Mycomp << 'END'
+cat > Mycomp << END
#!/bin/sh
-case " "$* in
+case " \$* " in
*\ -c*\ -o* | *\ -o*\ -c*)
exit 1
;;
esac
-exec gcc "$@"
+# Use '$CC', not 'gcc', to honour the compiler chosen by 'tests/defs'.
+exec $CC "\$@"
END
chmod +x Mycomp
CFLAGS=
export CFLAGS
-for conf in configure.1 configure.3; do
- cp $conf configure.in
-
- $ACLOCAL
- $AUTOCONF
- $AUTOMAKE --copy --add-missing
-
- rm -rf build
- mkdir build
- cd build
-
- # Make sure the compiler doesn't understand `-c -o'
- CC=`pwd`/../Mycomp
- export CC
-
- ../configure
- $MAKE
-
- cd ..
+# Make sure the compiler doesn't understand `-c -o'
+CC=`pwd`/Mycomp
+export CC
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE --copy --add-missing
+
+for vpath in : false; do
+ if $vpath; then
+ srcdir=..
+ mkdir build
+ cd build
+ else
+ srcdir=.
+ fi
+ $srcdir/configure
+ $MAKE
+ cd $srcdir
done
:
$ACLOCAL
$AUTOCONF
AUTOMAKE_fails --copy --add-missing
-grep 'Makefile.am:2:.*per-target.*AM_PROG_CC_C_O' stderr
+grep '^Makefile\.am:2:.*per-target.*AM_PROG_CC_C_O' stderr
cat >Makefile.am <<EOF
echo 'AUTOMAKE_OPTIONS = subdir-objects' >> Makefile.am
AUTOMAKE_fails --copy --add-missing
-grep 'Makefile.am:2:.*subdir.*AM_PROG_CC_C_O' stderr
+grep '^Makefile\.am:2:.*subdir.*AM_PROG_CC_C_O' stderr
+
+:
cat >> configure.in << 'END'
AC_PROG_CC
AM_PROG_CC_C_O
+$CC --version; $CC -v; # For debugging.
AC_OUTPUT
END
}
END
-cat > Mycomp << 'END'
+cat > Mycomp << END
#!/bin/sh
-case " "$* in
+case " \$* " in
*\ -c*\ -o* | *\ -o*\ -c*)
exit 1
;;
esac
-exec gcc "$@"
+# Use '$CC', not 'gcc', to honour the compiler chosen by 'tests/defs'.
+exec $CC "\$@"
END
chmod +x Mycomp
cat stderr >&2
grep 'mv.*the same file' stderr && Exit 1
-Exit 0
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Simple Tests support: redirection of file descriptors with
+# AM_TESTS_FD_REDIRECT.
+# See also related test 'parallel-tests-fd-redirect.test'.
+
+. ./defs || Exit 1
+
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
+
+cat > Makefile.am <<'END'
+TESTS = foo.test
+AM_TESTS_FD_REDIRECT = 3<three 4>four 5>>five 7<&0 8>&1 9>&2
+END
+
+echo '3333' > three
+chmod a-w three
+
+: > foo.test
+chmod a+x foo.test
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+./configure
+
+do_check ()
+{
+ cat foo.test # For debugging.
+ echo 'this line will be removed' > four
+ echo 'this line will not be removed' > five
+ st=0
+ echo 'ok ok ok' | $MAKE check >stdout 2>stderr || st=1
+ cat stdout
+ cat stderr >&2
+ cat four
+ test x"$parallel_tests" != x"yes" || cat foo.log
+ test $st -eq 0
+ grep '[ /]foo\.test: foofoofoo$' stdout
+ grep '[ /]foo\.test: barbarbar$' stderr
+ grep 'this line' four && Exit 1
+ grep '^3333$' four
+ grep '^this line will not be removed$' five
+ grep '^ok ok ok$' five
+ $EGREP '(foofoofoo|barbarbar|3333|ok ok ok|this line)' foo.log && Exit 1
+ :
+}
+
+# Try using both shell script and a perl script as the test, for
+# better coverage.
+
+cat > foo.test <<'END'
+#! /bin/sh
+set -e
+
+read FOO <&3
+test 3333 -eq "$FOO"
+echo "$FOO" >&4
+
+grep '^ok ok ok$' <&7 >&5
+
+echo " " $0: foofoofoo >&8
+echo " " $0: barbarbar >&9
+END
+
+do_check
+
+echo "#! $PERL -w" > foo.test
+cat >> foo.test <<'END'
+use warnings FATAL => 'all';
+use strict;
+
+open (FD3, "<&=3") or die "opening FD3: $!";
+open (FD4, ">&=4") or die "opening FD4: $!";
+open (FD5, ">&=5") or die "opening FD5: $!";
+open (FD7, "<&=7") or die "opening FD7: $!";
+open (FD8, ">&=8") or die "opening FD8: $!";
+open (FD9, ">&=9") or die "opening FD9: $!";
+
+chomp (my $FOO = <FD3>);
+die "$FOO != 3333" if not $FOO eq "3333";
+print FD4 "$FOO\n";
+
+chomp ($_ = <FD7>);
+die "$_ != 'ok ok ok'" if not $_ eq 'ok ok ok';
+print FD5 "$_\n";
+
+print FD8 " $0: foofoofoo\n";
+print FD9 " $0: barbarbar\n";
+END
+
+do_check
+
+:
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Helper testcase which generate input data for the other test
-# `instspc-*.test'. It basically delegates the work to the helper
-# script `instspc-test.sh'.
+# Check that auxiliary script 'test-driver' doesn't get needlessly
+# installed or referenced when the 'parallel-tests' option is not
+# used.
-# Ensure proper definition of $testsrcdir.
-# FIXME: we employ useless quoting below to please maintainer-check.
-. ./defs-static || 'exit' 99
-test -n "$testsrcdir" || 'exit' 99 # Sanity check.
+parallel_tests=no
+. ./defs || Exit 1
-instspc_action=generate-data
-. $testsrcdir/instspc-tests.sh
+echo 'TESTS = foo.test' > Makefile.am
+
+$ACLOCAL
+
+for opts in '' '-a' '--add-missing --copy'; do
+ $AUTOMAKE $opts
+ $FGREP 'test-driver' Makefile.in && Exit 1
+ find . | $FGREP 'test-driver' && Exit 1
+done
+
+:
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Check @substituted@ TESTS.
-# Note that in this test, we rely on the .test extension for the
-# substituted names: this is necessary for parallel-tests.
+# Check @substituted@ TESTS, some of which are also PROGRAMS.
+# See also sister test `check-subst.test'.
+required='cc native'
. ./defs || Exit 1
cat >> configure.in << 'END'
AC_PROG_CC
-AC_SUBST([script_tests], ['subst-pass-script.test subst-xfail-script.test'])
-AC_SUBST([prog_tests], ['subst-pass-prog.test$(EXEEXT) subst-xfail-prog.test$(EXEEXT)'])
-AC_SUBST([xfail_tests], ['xfail-script.test subst-xfail-script.test xfail-prog$(EXEEXT) subst-xfail-prog.test$(EXEEXT)'])
+AC_SUBST([script_tests],
+ ['subst-pass-script.test subst-xfail-script.test'])
+AC_SUBST([prog_tests],
+ ['subst-pass-prog.test$(EXEEXT) subst-xfail-prog.test$(EXEEXT)'])
+AC_SUBST([xfail_tests],
+ ['xfail-script.test subst-xfail-script.test]dnl
+ [ xfail-prog$(EXEEXT) subst-xfail-prog.test$(EXEEXT)'])
AC_OUTPUT
END
EXTRA_PROGRAMS = subst-pass-prog.test subst-xfail-prog.test
END
-cat >>pass-script.test <<'END'
+cat > pass-script.test <<'END'
#! /bin/sh
exit 0
END
-cat >>xfail-script.test <<'END'
+cat > xfail-script.test <<'END'
#! /bin/sh
exit 1
END
cp pass-script.test subst-pass-script.test
cp xfail-script.test subst-xfail-script.test
-cat >>pass-prog.c <<'END'
-int main() { return 0; }
+cat > pass-prog.c <<'END'
+int main (void) { return 0; }
END
-cat >>xfail-prog.c <<'END'
+cat > xfail-prog.c <<'END'
#include <stdlib.h>
-int main() { return EXIT_FAILURE; }
+int main (void) { return EXIT_FAILURE; }
END
# The .test extension is removed for the default source file name:
cp pass-prog.c subst-pass-prog.c
$MAKE all
$MAKE check
$MAKE distclean
+
:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2008, 2009, 2010, 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check @substituted@ TESTS.
+# Note that in this test, we rely on the .test extension for the
+# substituted names: this is necessary for parallel-tests.
+# See also sister test `check-subst-prog.test'.
+
+. ./defs || Exit 1
+
+cat >> configure.in << 'END'
+AC_SUBST([script_tests], ['subst-pass-script.sh subst-xfail-script.sh'])
+AC_SUBST([xfail_tests], ['xfail-script.test subst-xfail-script.sh'])
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+TESTS = pass-script.test xfail-script.test @script_tests@
+XFAIL_TESTS = @xfail_tests@
+END
+
+if test "$parallel_tests" = yes; then
+ unindent >> Makefile.am <<'END'
+ TEST_EXTENSIONS = .sh .test
+ SH_LOG_COMPILER = $(SHELL)
+END
+fi
+
+cat > pass-script.test <<'END'
+#! /bin/sh
+exit 0
+END
+cat > xfail-script.test <<'END'
+#! /bin/sh
+exit 1
+END
+cp pass-script.test subst-pass-script.sh
+cp xfail-script.test subst-xfail-script.sh
+chmod a+x pass-script.test xfail-script.test
+
+if test "$parallel_tests" != yes; then
+ chmod a+x subst-pass-script.sh subst-xfail-script.sh
+fi
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+for vpath in false : ; do
+
+ if $vpath; then
+ mkdir build
+ cd build
+ srcdir=..
+ else
+ srcdir=.
+ fi
+
+ $srcdir/configure
+ $MAKE all
+ $MAKE check
+ if test "$parallel_tests" = yes; then
+ ls -l
+ test -f pass-script.log
+ test -f xfail-script.log
+ test -f subst-pass-script.log
+ test -f subst-xfail-script.log
+ fi
+ $MAKE distclean
+ cd $srcdir
+
+done
+
+:
TESTS = frob.test
END
+test x"$parallel_tests" != x"yes" || : > test-driver
+
: > frob.test
$ACLOCAL
$AUTOMAKE
grep '^check-TESTS:' Makefile.in
-grep '^check-DEJAGNU' Makefile.in && Exit 1
+grep 'check-DEJAGNU' Makefile.in && Exit 1
-# check-TESTS is phony.
-sed -n '/^\.PHONY:/,/^$/p' Makefile.in | grep check-TESTS
+# 'check-TESTS' is phony.
+sed -n '/^\.PHONY:/,/^$/p' Makefile.in | $EGREP '(^| )check-TESTS($| )'
-# check should depend directly on check-am (similar tests are
+# 'check' should depend directly on 'check-am' (similar tests are
# in check2.test and check3.test).
-grep '^check: check-am' Makefile.in
+$EGREP '^check:.* check-am( |$)' Makefile.in
+
+:
# Check singular and plural in test summaries.
+# This test only makes sense for the older serial testsuite driver.
+parallel_tests=no
. ./defs || Exit 1
cat >> configure.in << 'END'
XFAIL_TESTS = xfail xpass xfail2 xpass2
END
-cat >>pass <<'END'
+cat > pass <<'END'
#! /bin/sh
exit 0
END
-cat >>fail <<'END'
+cat > fail <<'END'
#! /bin/sh
exit 1
END
-cat >>skip <<'END'
+cat > skip <<'END'
#! /bin/sh
exit 77
END
$ACLOCAL
$AUTOCONF
-$AUTOMAKE
+$AUTOMAKE -a
./configure
(
env TESTS='pass skip xfail' $MAKE -e check
$MAKE check
:
-) >stdout
+) >stdout || { cat stdout; Exit 1; }
cat stdout
grep '1 [tT]ests' stdout && Exit 1
TESTS = skip skip2
END
-cat >>skip <<'END'
+cat > skip <<'END'
#! /bin/sh
exit 77
END
$AUTOMAKE -a
./configure
-env TESTS=skip $MAKE -e check >stdout
-cat stdout
-grep '1.*passed' stdout && Exit 1
-env TESTS="skip skip2" $MAKE -e check >stdout
+env TESTS=skip $MAKE -e check >stdout || { cat stdout; Exit 1; }
+cat stdout
+if test x"$parallel_tests" = x"yes"; then
+ count_test_results total=1 pass=0 fail=0 skip=1 xfail=0 xpass=0 error=0
+else
+ grep '1.*passed' stdout && Exit 1
+ : For shells with buggy 'set -e'.
+fi
+
+env TESTS="skip skip2" $MAKE -e check >stdout || { cat stdout; Exit 1; }
cat stdout
-grep '2.*passed' stdout && Exit 1
+if test x"$parallel_tests" = x"yes"; then
+ count_test_results total=2 pass=0 fail=0 skip=2 xfail=0 xpass=0 error=0
+else
+ grep '2.*passed' stdout && Exit 1
+ : For shells with buggy 'set -e'.
+fi
:
cat > a.test << 'END'
#!/bin/sh
+echo a.test: exit status: ${A_EXIT_STATUS-0}
exit ${A_EXIT_STATUS-0}
END
cat > b.test << 'END'
#!/bin/sh
+echo b.test: exit status: ${B_EXIT_STATUS-0}
exit ${B_EXIT_STATUS-0}
END
chmod +x a.test b.test
test -f spanner.sum
# This checks will be run only by the autogenerated `check12-p.test'.
if test x"$parallel_tests" = x"yes"; then
- cat test-suite.log
- grep '^PASS: a\.test (exit: 0)' a.log
- grep '^PASS: b\.test (exit: 0)' b.log
+ test -f test-suite.log
+ test -f a.log
+ test -f b.log
else
:
fi
# This checks will be run only by the autogenerated `check12-p.test'.
if test x"$parallel_tests" = x"yes"; then
cat test-suite.log
- grep '^PASS: a\.test (exit: 0)' a.log
- grep '^FAIL: b\.test (exit: 1)' b.log
- grep '^FAIL: b\.test (exit: 1)' test-suite.log
+ cat a.log
+ cat b.log
+ grep '^a\.test: exit status: 0$' a.log
+ grep '^b\.test: exit status: 1$' b.log
+ grep '^FAIL: b$' test-suite.log
+ grep '^b\.test: exit status: 1$' test-suite.log
+ grep '^a\.test' test-suite.log && Exit 1
else :; fi
CHECKLOCAL_EXIT_STATUS=1 $MAKE check && Exit 1
# This checks will be run only by the autogenerated `check12-p.test'.
if test x"$parallel_tests" = x"yes"; then
cat test-suite.log
- grep '^PASS: a\.test (exit: 0)' a.log
- grep '^FAIL: b\.test (exit: 23)' b.log
- grep '^FAIL: b\.test (exit: 23)' test-suite.log
+ cat a.log
+ cat b.log
+ grep '^a\.test: exit status: 0$' a.log
+ grep '^b\.test: exit status: 23$' b.log
+ grep '^FAIL: b$' test-suite.log
+ grep '^b\.test: exit status: 23$' test-suite.log
+ grep '^a\.test' test-suite.log && Exit 1
else :; fi
grep 'check-local failed :-(' local.log
CLEANFILES = echo.sh
END
+if test x"$parallel_tests" = x"yes"; then
+ cp "$top_testsrcdir"/lib/test-driver .
+fi
+
$ACLOCAL
$AUTOCONF
$AUTOMAKE
./configure
-$MAKE check >stdout
+
+$MAKE check >stdout || { cat stdout; Exit 1; }
cat stdout
-grep 'PASS: subrun.sh' stdout
+grep '^PASS: subrun\.sh *$' stdout
+grep 'PASS.*echo\.sh' stdout && Exit 1
-# check should depend directly on check-am (similar tests are
+# 'check' should depend directly on 'check-am' (similar tests are
# in check.test and check3.test).
-grep 'check: check-recursive' Makefile.in
-grep 'check: check-am' dir/Makefile.in
+$EGREP '^check:.* check-recursive( |$)' Makefile.in
+$EGREP '^check:.* check-am( |$)' dir/Makefile.in
# Make sure subrun.sh is still on its line as above. This means Automake
# hasn't rewritten the TESTS line unnecessarily (we can tell, because all
# Automake variables are reformatted by VAR_PRETTY).
-grep ' subrun.sh' Makefile.in
+grep '^ subrun\.sh$' Makefile.in
+
+:
$ACLOCAL
$AUTOCONF
-$AUTOMAKE
+$AUTOMAKE -a
./configure --prefix "`pwd`/inst"
-$MAKE check >stdout
+
+$MAKE check >stdout || { cat stdout; Exit 1; }
cat stdout
-grep 'PASS: subrun.sh' stdout
+grep '^PASS: subrun\.sh *$' stdout
+grep 'PASS.*echo\.sh' stdout && Exit 1
-# check should depend directly on check-am (similar tests are
-# in check.test and check2.test).
-$FGREP 'check: $(BUILT_SOURCES)' Makefile.in
-$FGREP 'check: $(BUILT_SOURCES)' dir/Makefile.in
+# check should depend directly on $(BUILT_SOURCES) (similar tests
+# are in check.test and check2.test).
+$EGREP '^check:.* \$\(BUILT_SOURCES\)( |$)' Makefile.in
+$EGREP '^check:.* \$\(BUILT_SOURCES\)( |$)' dir/Makefile.in
$MAKE clean
# Sanity checks
$MAKE install
test -f command1.inc
test -f dir/command2.inc
+
+:
$ACLOCAL
$AUTOCONF
-$AUTOMAKE
+$AUTOMAKE --add-missing
+
./configure --prefix "`pwd`/inst"
+
$MAKE check >stdout && { cat stdout; Exit 1; }
cat stdout
-grep 'FAIL: fail.sh' stdout
-grep 'PASS: ok.sh' stdout && Exit 1
+grep '^FAIL: fail\.sh *$' stdout
+grep '^PASS: ok\.sh *$' stdout && Exit 1
# The exit status of `make -k' can be anything
# (depending on the Make implementation)
$MAKE -k check >stdout || :
cat stdout
-grep 'FAIL: fail.sh' stdout
-grep 'PASS: ok.sh' stdout
+grep '^FAIL: fail\.sh *$' stdout
+grep '^PASS: ok\.sh *$' stdout
-# should also works when -k is not in first position
+# Should also works when -k is not in first position.
$MAKE -s -k check >stdout || :
cat stdout
-grep 'FAIL: fail.sh' stdout
-grep 'PASS: ok.sh' stdout
+grep '^FAIL: fail\.sh *' stdout
+grep '^PASS: ok\.sh *' stdout
# The rest of the test is for GNU Make.
-# Use --version AND -v, because SGI Make doesn't fail on --version.
-# Also grep for GNU because newer versions of FreeBSD make do
-# not complain about `--version' (they seem to silently ignore it).
-($MAKE --version -v | grep GNU) || Exit 0
+if using_gmake; then
+ # Try with a long-option that do not have a short option equivalent
+ # (here, --no-print-directory). That should cause all options to
+ # appear verbatim in MAKEFLAGS.
+ $MAKE --no-print-directory -k check >stdout || :
+ cat stdout
+ grep '^FAIL: fail\.sh *$' stdout
+ grep '^PASS: ok\.sh *$' stdout
+fi
-# Try with a long-option that do not have a short option equivalent
-# (here, --no-print-directory). That should cause all options to
-# appear verbatim in MAKEFLAGS.
-$MAKE --no-print-directory -k check >stdout || :
-cat stdout
-grep 'FAIL: fail.sh' stdout
-grep 'PASS: ok.sh' stdout
+:
# Test TESTS = $(check_PROGRAMS)
+required='cc native'
. ./defs || Exit 1
cat >> configure.in << 'END'
test -f one$(EXEEXT)
test -f two$(EXEEXT)
touch ok
+.PHONY: print-tests
print-tests:
echo BEG: $(TESTS) :END
END
$AUTOCONF
$AUTOMAKE -a
-cat >one.c <<END
-int main() { return 0; }
+cat > one.c << 'END'
+int main (void)
+{
+ return 0;
+}
END
cp one.c two.c
./configure
$MAKE check
test -f ok
-EXEEXT=.bin $MAKE -e print-tests >output
-cat output
+EXEEXT=.bin $MAKE -e print-tests >stdout || { cat stdout; Exit 1; }
+cat stdout
+$FGREP 'BEG: one.bin two.bin :END' stdout
# No am__EXEEXT_* variable is needed.
grep '_EXEEXT_[1-9]' Makefile.in && Exit 1
-grep 'BEG: one.bin two.bin :END' output
$FGREP 'TESTS = $(check_PROGRAMS)' Makefile.in
+
+:
. ./defs || Exit 1
cat >> configure.in << 'END'
-AC_PROG_CC
AC_OUTPUT
END
cd build
../configure
$MAKE check
+
+:
# Check EXEEXT extension for XFAIL_TESTS.
+required=cc
. ./defs || Exit 1
cat >> configure.in << 'END'
XFAIL_TESTS = a b c d
check_PROGRAMS = a c d
check_SCRIPTS = b
+EXTRA_DIST = $(check_SCRIPTS)
+.PHONY: print-xfail-tests
print-xfail-tests:
@echo BEG: $(XFAIL_TESTS) :END
END
-cat >>b <<'END'
+cat > b <<'END'
#! /bin/sh
exit 1
END
chmod a+x b
-cat >>a.c <<'END'
+cat > a.c <<'END'
#include <stdlib.h>
-int main() { return EXIT_FAILURE; }
+int main (void)
+{
+ return EXIT_FAILURE;
+}
END
cp a.c c.c
./configure
$MAKE check
-EXEEXT=.bin $MAKE -e print-xfail-tests >foo
-cat foo
-grep 'BEG: a.bin b c.bin d.bin :END' foo
+EXEEXT=.bin $MAKE -e print-xfail-tests >stdout || { cat stdout; Exit 1; }
+cat stdout
+$FGREP 'BEG: a.bin b c.bin d.bin :END' stdout
-$MAKE distclean
-mkdir build
-cd build
-../configure
-$MAKE check
+$MAKE distcheck
+
+:
# Check subdir TESTS.
+required='cc native'
. ./defs || Exit 1
cat >> configure.in << 'END'
mkdir sub
-cat >>foo <<'END'
+cat > foo <<'END'
#! /bin/sh
test -f "$srcdir/Makefile.am"
END
-cat >>sub/foo <<'END'
+cat > sub/foo <<'END'
#! /bin/sh
test -f "$srcdir/Makefile.am"
END
chmod a+x foo sub/foo
-cat >>bar.c <<'END'
-int main() { return 0; }
+cat > bar.c <<'END'
+int main (void) { return 0; }
END
-cat >>sub/bar.c <<'END'
-int main() { return 0; }
+cat > sub/bar.c <<'END'
+int main (void) { return 0; }
END
-cat >>baz.c <<'END'
+cat > baz.c <<'END'
#include <stdlib.h>
-int main() { return EXIT_FAILURE; }
+int main (void) { return EXIT_FAILURE; }
END
-cat >>sub/baz.c <<'END'
+cat > sub/baz.c <<'END'
#include <stdlib.h>
-int main() { return EXIT_FAILURE; }
+int main (void) { return EXIT_FAILURE; }
END
$ACLOCAL
grep '^[^X]*PASS.* sub/bar' stdout
grep '^[^X]*FAIL.* baz' stdout
grep 'XFAIL.* sub/baz' stdout
-# parallel-tests should not add circular dependencies.
+# `parallel-tests' should not add circular dependencies.
# Look for known warnings from a couple of `make' implementations.
grep -i 'circular.*dependency' stderr && Exit 1
grep -i 'graph cycles' stderr && Exit 1
grep '^[^X]*PASS.*sub/bar' stdout
grep '^[^X]*FAIL.*baz' stdout
grep 'XFAIL.*sub/baz' stdout
+
:
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Make sure distclean works in cygnus mode.
-# Report from Daniel Jacobowitz
+# Report from Daniel Jacobowitz.
. ./defs || Exit 1
$AUTOCONF
./configure
-test -f Makefile # sanity check
+test -f Makefile # Sanity check.
$MAKE dummy
# Again, make sure that the generated Makefile do not unduly
testmk:
grep '$str' Makefile
END
-$MAKE Makefile # for non-GNU make
+$MAKE Makefile # For non-GNU make.
$MAKE testam testin testmk
$MAKE distcheck
$AUTOCONF
./configure
-test -f Makefile # sanity check
+test -f Makefile # Sanity check.
$MAKE dummy
# Again, make sure that the generated Makefile do not unduly
END
echo "# $str2" >> two.in
echo "# $str3" >> three.in
-$MAKE Makefile # for non-GNU make
+$MAKE Makefile # For non-GNU make.
$MAKE testam testin testmk test2 test3
$MAKE distcheck
./configure
-grep '=GrEpMe_am=' Makefile && Exit 1 # sanity check
-grep '=GrEpMe_dep=' Makefile && Exit 1 # likewise
+grep '=GrEpMe_am=' Makefile && Exit 1 # Sanity check.
+grep '=GrEpMe_dep=' Makefile && Exit 1 # Likewise.
$MAKE test-distcommon
$MAKE test-distdir
echo '# =GrEpMe_am=' >> Makefile.am
echo '# =GrEpMe_dep=' >> Makefile.dep
-$MAKE Makefile # for non-GNU make
+$MAKE Makefile # For non-GNU make.
$MAKE test-grep
$MAKE test-distcommon
$MAKE test-distdir
$srcdir/configure
- test -f demo/version.good # sanity check
- test ! -s demo/version.good # likewise
+ test -f demo/version.good # Sanity check.
+ test ! -s demo/version.good # Likewise.
cd demo
# Makefile should not depend on version.gin.
rm -f ../$srcdir/demo/version.gin
$MAKE Makefile
- test ! -r ../$srcdir/demo/version.gin # sanity check
+ test ! -r ../$srcdir/demo/version.gin # Sanity check.
# version.good should depend on version.gin.
rm -f version.good
# Try to verify that we errored out for the right reason.
$FGREP version.gin output
- cd .. # back in top builddir
+ cd .. # Back in top builddir.
cd $srcdir
# Re-create it for the next pass (if any).
: > Makefile.am
: > subdir/foo
cat > subdir/Makefile.am << 'END'
-.PHONY: test
+# DIST_COMMON should contain `foo', not `subdir/foo'.
test:
- ## DIST_COMMON should contain `foo', not `subdir/foo'.
case '$(DIST_COMMON)' in *subdir/foo*) exit 1;; *) exit 0;; esac
echo ' ' $(DIST_COMMON) ' ' | grep '[ /]foo '
+.PHONY: test
END
$ACLOCAL
TERM=ansi
export TERM
-red='\e[0;31m'
-grn='\e[0;32m'
-lgn='\e[1;32m'
-blu='\e[1;34m'
-std='\e[m'
+esc='\e'
+# Escape `[' for grep, below.
+red="$esc\[0;31m"
+grn="$esc\[0;32m"
+lgn="$esc\[1;32m"
+blu="$esc\[1;34m"
+mgn="$esc\[0;35m"
+std="$esc\[m"
# Check that grep can parse nonprinting characters.
# BSD 'grep' works from a pipe, but not a seekable file.
# GNU or BSD 'grep -a' works on files, but is not portable.
case `echo "$std" | grep .` in
"$std") ;;
- *) echo "$me: grep can't parse nonprinting characters" >&2; Exit 77;;
+ *) skip_ "grep can't parse nonprinting characters";;
esac
cat >>configure.in <<END
cat >Makefile.am <<'END'
AUTOMAKE_OPTIONS = color-tests
TESTS = $(check_SCRIPTS)
-check_SCRIPTS = pass fail skip xpass xfail
+check_SCRIPTS = pass fail skip xpass xfail error
XFAIL_TESTS = xpass xfail
END
exit 77
END
+cat >error <<END
+#! /bin/sh
+exit 99
+END
+
cp fail xfail
cp pass xpass
-chmod +x pass fail skip xpass xfail
+chmod +x pass fail skip xpass xfail error
$ACLOCAL
-$AUTOMAKE
$AUTOCONF
-./configure
+$AUTOMAKE --add-missing
test_color ()
{
# Not a useless use of cat; see above comments about grep.
- cat stdout | grep ": pass" | $FGREP "$grn"
- cat stdout | grep ": fail" | $FGREP "$red"
- cat stdout | grep ": xfail" | $FGREP "$lgn"
- cat stdout | grep ": xpass" | $FGREP "$red"
- cat stdout | grep ": skip" | $FGREP "$blu"
+ cat stdout | grep "^${grn}PASS${std}: .*pass"
+ cat stdout | grep "^${red}FAIL${std}: .*fail"
+ cat stdout | grep "^${blu}SKIP${std}: .*skip"
+ cat stdout | grep "^${lgn}XFAIL${std}: .*xfail"
+ cat stdout | grep "^${red}XPASS${std}: .*xpass"
+ # The old serial testsuite driver doesn't distinguish between failures
+ # and hard errors.
+ if test x"$parallel_tests" = x"yes"; then
+ cat stdout | grep "^${mgn}ERROR${std}: .*error"
+ else
+ cat stdout | grep "^${red}FAIL${std}: .*error"
+ fi
+ :
}
test_no_color ()
{
- # Not a useless use of cat; see above comments about grep.
- cat stdout | grep ": pass" | $FGREP "$grn" && Exit 1
- cat stdout | grep ": fail" | $FGREP "$red" && Exit 1
- cat stdout | grep ": xfail" | $FGREP "$lgn" && Exit 1
- cat stdout | grep ": xpass" | $FGREP "$red" && Exit 1
- cat stdout | grep ": skip" | $FGREP "$blu" && Exit 1
- :
+ # With make implementations that, like Solaris make, in case of errors
+ # print the whole failing recipe on standard output, we should content
+ # ourselves with a laxer check, to avoid false positives.
+ # Keep this in sync with lib/am/check.am:$(am__color_tests).
+ if $FGREP '= Xalways || test -t 1 ' stdout; then
+ # Extra verbose make, resort to laxer checks.
+ # Note that we also want to check that the testsuite summary is
+ # not unduly colorized.
+ (
+ set +e # In case some grepped regex below isn't matched.
+ # Not a useless use of cat; see above comments about grep.
+ cat stdout | grep "TOTAL.*:"
+ cat stdout | grep "PASS.*:"
+ cat stdout | grep "FAIL.*:"
+ cat stdout | grep "SKIP.*:"
+ cat stdout | grep "XFAIL.*:"
+ cat stdout | grep "XPASS.*:"
+ cat stdout | grep "ERROR.*:"
+ cat stdout | grep 'test.*expected'
+ cat stdout | grep 'test.*not run'
+ cat stdout | grep '===='
+ cat stdout | grep '[Ss]ee .*test-suite\.log'
+ cat stdout | grep '[Tt]estsuite summary'
+ ) | grep "$esc" && Exit 1
+ : For shells with broken 'set -e'
+ else
+ cat stdout | grep "$esc" && Exit 1
+ : For shells with broken 'set -e'
+ fi
}
-AM_COLOR_TESTS=always $MAKE -e check >stdout && { cat stdout; Exit 1; }
-cat stdout
-test_color
+for vpath in false :; do
+
+ if $vpath; then
+ mkdir build
+ cd build
+ srcdir=..
+ else
+ srcdir=.
+ fi
+
+ $srcdir/configure
+
+ AM_COLOR_TESTS=always $MAKE -e check >stdout && { cat stdout; Exit 1; }
+ cat stdout
+ test_color
+
+ $MAKE -e check >stdout && { cat stdout; Exit 1; }
+ cat stdout
+ test_no_color
+
+ $MAKE distclean
+ cd $srcdir
+
+done
+
+:
TERM=ansi
export TERM
-red='\e[0;31m'
-grn='\e[0;32m'
-lgn='\e[1;32m'
-blu='\e[1;34m'
-std='\e[m'
+esc='\e'
+# Escape `[' for grep, below.
+red="$esc\[0;31m"
+grn="$esc\[0;32m"
+lgn="$esc\[1;32m"
+blu="$esc\[1;34m"
+mgn="$esc\[0;35m"
+std="$esc\[m"
# Check that grep can parse nonprinting characters.
# BSD 'grep' works from a pipe, but not a seekable file.
# GNU or BSD 'grep -a' works on files, but is not portable.
case `echo "$std" | grep .` in
"$std") ;;
- *) echo "$me: grep can't parse nonprinting characters" >&2; Exit 77;;
+ *) skip_ "grep can't parse nonprinting characters";;
esac
# Check that we have a working expect program.
eval spawn $env(THE_SYSTEM_SHELL) -c :
expect eof
END
-THE_SYSTEM_SHELL=/bin/sh expect -f expect-check || {
- echo "$me: failed to find a working expect program" >&2
- Exit 77
-}
+THE_SYSTEM_SHELL=/bin/sh expect -f expect-check \
+ || skip_ "$me: failed to find a working expect program"
# Do the tests.
cat >Makefile.am <<'END'
AUTOMAKE_OPTIONS = color-tests
TESTS = $(check_SCRIPTS)
-check_SCRIPTS = pass fail skip xpass xfail
+check_SCRIPTS = pass fail skip xpass xfail error
XFAIL_TESTS = xpass xfail
END
exit 77
END
+cat >error <<END
+#! /bin/sh
+exit 99
+END
+
cp fail xfail
cp pass xpass
-chmod +x pass fail skip xpass xfail
+chmod +x pass fail skip xpass xfail error
$ACLOCAL
-$AUTOMAKE
$AUTOCONF
-./configure
+$AUTOMAKE --add-missing
test_color ()
{
# Not a useless use of cat; see above comments about grep.
- cat stdout | grep ": pass" | $FGREP "$grn"
- cat stdout | grep ": fail" | $FGREP "$red"
- cat stdout | grep ": xfail" | $FGREP "$lgn"
- cat stdout | grep ": xpass" | $FGREP "$red"
- cat stdout | grep ": skip" | $FGREP "$blu"
+ cat stdout | grep "^${grn}PASS${std}: .*pass"
+ cat stdout | grep "^${red}FAIL${std}: .*fail"
+ cat stdout | grep "^${blu}SKIP${std}: .*skip"
+ cat stdout | grep "^${lgn}XFAIL${std}: .*xfail"
+ cat stdout | grep "^${red}XPASS${std}: .*xpass"
+ # The old serial testsuite driver doesn't distinguish between failures
+ # and hard errors.
+ if test x"$parallel_tests" = x"yes"; then
+ cat stdout | grep "^${mgn}ERROR${std}: .*error"
+ else
+ cat stdout | grep "^${red}FAIL${std}: .*error"
+ fi
+ :
}
test_no_color ()
{
- # Not a useless use of cat; see above comments about grep.
- cat stdout | grep ": pass" | $FGREP "$grn" && Exit 1
- cat stdout | grep ": fail" | $FGREP "$red" && Exit 1
- cat stdout | grep ": xfail" | $FGREP "$lgn" && Exit 1
- cat stdout | grep ": xpass" | $FGREP "$red" && Exit 1
- cat stdout | grep ": skip" | $FGREP "$blu" && Exit 1
- :
+ # With make implementations that, like Solaris make, in case of errors
+ # print the whole failing recipe on standard output, we should content
+ # ourselves with a laxer check, to avoid false positives.
+ # Keep this in sync with lib/am/check.am:$(am__color_tests).
+ if $FGREP '= Xalways || test -t 1 ' stdout; then
+ # Extra verbose make, resort to laxer checks.
+ # Note that we also want to check that the testsuite summary is
+ # not unduly colorized.
+ (
+ set +e # In case some grepped regex below isn't matched.
+ # Not a useless use of cat; see above comments about grep.
+ cat stdout | grep "TOTAL.*:"
+ cat stdout | grep "PASS.*:"
+ cat stdout | grep "FAIL.*:"
+ cat stdout | grep "SKIP.*:"
+ cat stdout | grep "XFAIL.*:"
+ cat stdout | grep "XPASS.*:"
+ cat stdout | grep "ERROR.*:"
+ cat stdout | grep 'test.*expected'
+ cat stdout | grep 'test.*not run'
+ cat stdout | grep '===='
+ cat stdout | grep '[Ss]ee .*test-suite\.log'
+ cat stdout | grep '[Tt]estsuite summary'
+ ) | grep "$esc" && Exit 1
+ : For shells with broken 'set -e'
+ else
+ cat stdout | grep "$esc" && Exit 1
+ : For shells with broken 'set -e'
+ fi
}
cat >expect-make <<'END'
expect eof
END
-MAKE=$MAKE expect -f expect-make >stdout \
- || { cat stdout; Exit 1; }
-cat stdout
-test_color
+for vpath in false :; do
+
+ if $vpath; then
+ mkdir build
+ cd build
+ srcdir=..
+ else
+ srcdir=.
+ fi
+
+ $srcdir/configure
+
+ MAKE=$MAKE expect -f $srcdir/expect-make >stdout \
+ || { cat stdout; Exit 1; }
+ cat stdout
+ test_color
+
+ AM_COLOR_TESTS=no MAKE=$MAKE expect -f $srcdir/expect-make >stdout \
+ || { cat stdout; Exit 1; }
+ cat stdout
+ test_no_color
+
+ $MAKE distclean
+ cd $srcdir
+
+done
-AM_COLOR_TESTS=no MAKE=$MAKE expect -f expect-make >stdout \
- || { cat stdout; Exit 1; }
-cat stdout
-test_no_color
+:
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Test for PR/322.
+# Automake 1.6.1 seems to have a problem parsing comments that use
+# '\' to span multiple lines.
. ./defs || Exit 1
$MAKE
grep '# SOME_FILES' Makefile
grep '# *file3' Makefile
+
+:
7.test
EOF
+: > test-driver
+
$ACLOCAL
$AUTOMAKE
$ACLOCAL
$AUTOMAKE
-grep TEX Makefile.in # for debugging
+grep TEX Makefile.in # For debugging.
grep '^TEXINFO_TEX *= *tex/texinfo\.tex *# some comment w/ a slash *$' Makefile.in
$AUTOCONF
. ./defs || Exit 1
-cp "$top_testsrcdir/lib/compile" .
+get_shell_script compile
# -o 'a c' should not be stripped because 'a c' is not an object
# (it does not matter whether touch creates ./-- or not)
. ./defs || Exit 1
-cp "$top_testsrcdir/lib/compile" .
+get_shell_script compile
cat >mycc <<'END'
source_seen=no
# Absolute w32 paths should be accepted.
# Do not actually run this test on anything that could be w32.
-test -d "C:\\" && Exit 77
-case $PATH_SEPARATOR in ';'|':');; *) Exit 77;; esac
+if test -d 'C:\'; then
+ skip_ "this test shouldn't run on a win32-like system"
+fi
+case $PATH_SEPARATOR in
+ ';'|':');;
+ *) skip_ "unrecognized PATH separator \`$PATH_SEPARATOR'"
+esac
amtest_source='C:\libltdl\libltdl\slist.c'
amtest_object='C:\libltdl\libltdl\libltdl_libltdl_la-slist.obj'
# Make sure `compile' wraps the Microsoft C/C++ compiler (cl) correctly
-required=xsi-shell
+required=xsi-lib-shell
. ./defs || Exit 1
-cp "$top_testsrcdir/lib/compile" .
+get_shell_script compile
# Use a dummy cl, since cl isn't readily available on all systems
cat >cl <<'END'
required='cl'
. ./defs || Exit 1
+get_shell_script compile
+
mkdir sub
cat >sub/foo.c <<'EOF'
. ./defs || Exit 1
-cp "$top_testsrcdir/lib/compile" .
+get_shell_script compile
# Use a dummy cl, since cl isn't readily available on all systems
cat >cl <<'END'
: >Makefile.am
+# This will be sourced, nor executed.
cat >check_host.in << 'END'
-#! /bin/sh
-case @host_os@ in
+case '@host_os@' in
mingw*)
;;
*)
- exit 77
+ skip_ "target OS is not MinGW"
;;
esac
case @build_os@ in
mingw* | cygwin*)
;;
*)
- winepath -w / || exit 77
+ winepath -w / \
+ || skip_ "not on MinGW or Cygwin, and winepath not available"
;;
esac
END
$AUTOCONF
$AUTOMAKE -a
./configure
-./check_host
+. ./check_host
pwd=`pwd`
# Make sure `compile' searches libraries correctly
-required=xsi-shell
+required=xsi-lib-shell
. ./defs || Exit 1
-cp "$top_testsrcdir/lib/compile" .
+get_shell_script compile
# Use a dummy cl, since cl isn't readily available on all systems
cat >cl <<'END'
. ./defs || Exit 1
cat >> configure.in << 'END'
-AM_CONDITIONAL(TEST, true)
+AM_CONDITIONAL([TEST], [true])
AC_OUTPUT
END
grep '^TEST_TRUE' Makefile.in && Exit 1
grep '^@TEST_TRUE@VAR = true$' Makefile.in
grep '^@TEST_FALSE@VAR = false$' Makefile.in
+
+:
cat >> configure.in << 'END'
AC_PROG_CC
-AM_CONDITIONAL(USE_A,[test x = y])
-AM_CONDITIONAL(USE_B,[test x = z])
+AM_CONDITIONAL([USE_A], [test x = y])
+AM_CONDITIONAL([USE_B], [test x = z])
AC_OUTPUT
END
$ACLOCAL
$AUTOMAKE -a
grep 'USE_A_FALSE.*USE_B_FALSE.*output_c\...OBJEXT.' Makefile.in
+
+:
. ./defs || Exit 1
cat >> configure.in << 'END'
-AC_PROG_CC
+AC_SUBST([CC], [false])
AM_CONDITIONAL([USE_A], [test -z "$two"])
AC_SUBST([SUBSTVAR], [bar])
AC_OUTPUT
END
cat > Makefile.am << 'END'
-
if USE_A
foolibs=faz.la
else
foo_SOURCES = foo.c
LDADD = $(SUBSTVAR) $(foolibs)
-print:
- @echo BEG: $(foo_DEPENDENCIES) :END
+.PHONY: test1 test2
+test1:
+ test faz.la = $(foo_DEPENDENCIES)
+test2:
+ test -z "`echo $(foo_DEPENDENCIES)`"
END
: > config.guess
$ACLOCAL
$AUTOCONF
-$AUTOMAKE
+$AUTOMAKE --ignore-deps
./configure
-$MAKE -e print > stdout
-cat stdout
-grep 'BEG: faz.la :END' stdout
+$MAKE test1
./configure two=yes
-$MAKE -e print > stdout
-cat stdout
-grep 'BEG: :END' stdout
+$MAKE test2
+
+:
AC_PROG_CC
AC_PROG_CXX
AC_PROG_RANLIB
-AM_CONDITIONAL(COND1, true)
-AM_CONDITIONAL(COND2, true)
+AM_CONDITIONAL([COND1], [true])
+AM_CONDITIONAL([COND2], [true])
END
cat > Makefile.am << 'END'
$ACLOCAL
$AUTOMAKE
-grep 'am_libtest_a_OBJECTS = .*$' Makefile.in
+grep '^am_libtest_a_OBJECTS =' Makefile.in
+
+:
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Test for bug in conditionals.
-# Report from Robert Boehne
+# Report from Robert Boehne.
. ./defs || Exit 1
END
cat > Makefile.am << 'END'
-
if COND1
BUILD_helldl = helldl
helldl_SOURCES = dlmain.c
$ACLOCAL
$AUTOMAKE
-num=`grep 'helldl$(EXEEXT):' Makefile.in | wc -l`
-test $num -eq 2
+$FGREP helldl Makefile.in # For debugging.
+test `$FGREP -c 'helldl$(EXEEXT):' Makefile.in` -eq 2
:
END
cat > Makefile.am << 'END'
-
if COND1
if COND2
bin_SCRIPTS = helldl
$ACLOCAL
$AUTOMAKE
+$FGREP helldl Makefile.in # For debugging.
num1=`$FGREP 'helldl$(EXEEXT):' Makefile.in | wc -l`
num2=`$FGREP '@COND1_FALSE@@COND2_TRUE@helldl$(EXEEXT):' Makefile.in | wc -l`
test $num1 -eq 4
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Test for bug in conditionals in SOURCES with variable substitution references.
-# Report from Richard Boulton
+# Test for bug in conditionals in SOURCES with variable substitution
+# references.
+# Report from Richard Boulton.
-required=gcc
. ./defs || Exit 1
-cat > configure.in << 'END'
-AC_INIT(hello.c)
-AM_INIT_AUTOMAKE(hello,0.23)
-AC_PROG_CC
-AM_CONDITIONAL(COND1, true)
-AC_OUTPUT(Makefile)
+cat >> configure.in << 'END'
+AM_CONDITIONAL([COND1], [true])
+AC_OUTPUT
END
-cat > hello.c << 'END'
-END
+: > hello.c
cat > Makefile.am << 'END'
+AUTOMAKE_OPTIONS = no-dependencies
+CC = false
+OBJEXT = o
if COND1
var = foo.c
bin_PROGRAMS = hell
hell_SOURCES = $(var:=)
-echorule:
- @echo $(hell_SOURCES) $(hell_OBJECTS)
-
+.PHONY: test
+test:
+ test "`echo $(hell_SOURCES) $(hell_OBJECTS)`" = "foo.c foo.o"
END
-# Ignore user CFLAGS.
-CFLAGS=
-export CFLAGS
-
$ACLOCAL
$AUTOCONF
$AUTOMAKE -a
-
./configure
+$MAKE test
-val=`$MAKE -s echorule`;
-echo $val
-test "x$val" = "xfoo.c foo.o"
+:
# Test for being able to define an object to be generated in different ways
# according to a conditional.
-# Report from Richard Boulton
+# Report from Richard Boulton.
. ./defs || Exit 1
-cat > configure.in << 'END'
-AC_INIT(Makefile.am)
-AM_INIT_AUTOMAKE(hello,0.23)
+cat >> configure.in << 'END'
AC_PROG_CC
AC_PROG_CXX
-AM_CONDITIONAL(COND1, true)
-AC_OUTPUT(Makefile)
+AM_CONDITIONAL([COND1], [true])
+AC_OUTPUT
END
cat > Makefile.am << 'END'
$ACLOCAL
$AUTOMAKE -a
+
+:
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Regression test for substitution references to conditional variables.
-# Report from Richard Boulton
+# Report from Richard Boulton.
-required='GNUmake gcc'
. ./defs || Exit 1
-cat > configure.in << 'END'
-AC_INIT(Makefile.am)
-AM_INIT_AUTOMAKE(hello,0.23)
-AC_PROG_CC
-AM_CONDITIONAL(COND1, true)
-AM_CONDITIONAL(COND2, true)
-AC_OUTPUT(Makefile)
+cat >> configure.in << 'END'
+AM_CONDITIONAL([COND1], [true])
+AM_CONDITIONAL([COND2], [true])
+AC_OUTPUT
END
cat > Makefile.am << 'END'
+AUTOMAKE_OPTIONS = no-dependencies
+CC = false
+OBJEXT = obj
var1 = dlmain
helldl_SOURCES = $(var3)
-echorule:
- @echo $(helldl_SOURCES) $(helldl_OBJECTS)
+.PHONY: test
+test:
+ test x"`echo $(helldl_SOURCES) $(helldl_OBJECTS)`" = \
+ x"dlmain.c foo.c dlmain.obj foo.obj"
bin_PROGRAMS = helldl
END
-# Ignore user CFLAGS.
-CFLAGS=
-export CFLAGS
-
$ACLOCAL
$AUTOCONF
$AUTOMAKE -a
-
./configure
+$MAKE test
-val=`$MAKE --no-print-directory echorule`;
-echo $val
-test "x$val" = "xdlmain.c foo.c dlmain.o foo.o"
+:
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Regression test for substitution references to conditional variables.
-# Report from Richard Boulton
+# Report from Richard Boulton.
-required='GNUmake gcc'
. ./defs || Exit 1
-cat > configure.in << 'END'
-AC_INIT(Makefile.am)
-AM_INIT_AUTOMAKE(hello,0.23)
-AC_PROG_CC
-AM_CONDITIONAL(COND1, test "x$CONDITION1" = "xtrue")
-AM_CONDITIONAL(COND2, test "x$CONDITION2" = "xtrue")
-AC_OUTPUT(Makefile)
+cat >> configure.in << 'END'
+AC_SUBST([CC], [false])
+AC_SUBST([OBJEXT], [o])
+AM_CONDITIONAL([COND1], [test "x$CONDITION1" = "xtrue"])
+AM_CONDITIONAL([COND2], [test "x$CONDITION2" = "xtrue"])
+AC_OUTPUT
END
cat > Makefile.am << 'END'
+bin_PROGRAMS = helldl
var1 = dlmain
helldl_SOURCES = $(var3:.c=1.c) $(var4:.c=2.c)
-echorule:
- @echo $(helldl_SOURCES) $(helldl_OBJECTS)
+got = `echo $(helldl_SOURCES) $(helldl_OBJECTS)`
-bin_PROGRAMS = helldl
+.PHONY: test
+test:
+ test x"$(exp)" = x"$(got)"
END
-# Ignore user CFLAGS.
-CFLAGS=
-export CFLAGS
-
$ACLOCAL
$AUTOCONF
-$AUTOMAKE -a
+$AUTOMAKE -a -i
CONDITION1=true CONDITION2=true ./configure
-vala=`$MAKE --no-print-directory echorule`;
+exp='dlmaina1.c dlmainb2.c dlmaina1.o dlmainb2.o' $MAKE -e test
CONDITION1=true CONDITION2=false ./configure
-valb=`$MAKE --no-print-directory echorule`;
+exp='dlmainb1.c dlmaina2.c dlmainb1.o dlmaina2.o' $MAKE -e test
CONDITION1=false CONDITION2=true ./configure
-valc=`$MAKE --no-print-directory echorule`;
+exp='dlmaina1.c dlmainb2.c dlmaina1.o dlmainb2.o' $MAKE -e test
CONDITION1=false CONDITION2=false ./configure
-vald=`$MAKE --no-print-directory echorule`;
+exp='dlmainb1.c dlmaina2.c dlmainb1.o dlmaina2.o' $MAKE -e test
-echo $vala
-echo $valb
-echo $valc
-echo $vald
-test "x$vala" = "xdlmaina1.c dlmainb2.c dlmaina1.o dlmainb2.o" || Exit 1
-test "x$valb" = "xdlmainb1.c dlmaina2.c dlmainb1.o dlmaina2.o" || Exit 1
-test "x$valc" = "xdlmaina1.c dlmainb2.c dlmaina1.o dlmainb2.o" || Exit 1
-test "x$vald" = "xdlmainb1.c dlmaina2.c dlmainb1.o dlmaina2.o" || Exit 1
+:
. ./defs || Exit 1
-cat > configure.in << 'END'
-AC_INIT
-AM_INIT_AUTOMAKE(nonesuch, nonesuch)
-AM_CONDITIONAL(TEST, true)
-AC_OUTPUT(Makefile)
+cat >> configure.in << 'END'
+AM_CONDITIONAL([TEST], [true])
END
cat > Makefile.am << 'END'
$ACLOCAL
AUTOMAKE_fails
-grep 'Makefile.am:4:.*dir2.*does not exist' stderr
+grep '^Makefile\.am:4:.*dir2.*does not exist' stderr
+
+:
. ./defs || Exit 1
-cat > configure.in << 'END'
-AC_INIT(Makefile.am)
-AM_INIT_AUTOMAKE(hello,0.23)
+cat >> configure.in << 'END'
AC_PROG_CC
-AM_CONDITIONAL(COND1, true)
-AC_OUTPUT(Makefile)
+AM_CONDITIONAL([COND1], [true])
END
cat > Makefile.am << 'END'
-
var1 = $(var2)
if COND1
$ACLOCAL
$AUTOCONF
AUTOMAKE_fails -a
-grep "recursively defined" stderr
+grep "variable.*var2.*recursively defined" stderr
+
+:
. ./defs || Exit 1
cat >> configure.in << 'END'
-AC_PROG_CC
-AM_CONDITIONAL(COND1, true)
-AM_CONDITIONAL(COND2, true)
-AM_CONDITIONAL(COND3, true)
+AM_CONDITIONAL([COND1], [true])
+AM_CONDITIONAL([COND2], [true])
+AM_CONDITIONAL([COND3], [true])
AC_OUTPUT
END
BAR += bar3
endif
+.PHONY: test
test:
@echo BAR: $(BAR) :BAR
@echo FOO: $(FOO) :FOO
./configure
$MAKE test | $FGREP 'BAR: bar12 bar bar3 :BAR'
$MAKE test | $FGREP 'FOO: foo foo1 foo2 foo1b :FOO'
+
+:
. ./defs || Exit 1
cat >> configure.in << 'END'
-AC_PROG_CC
-AM_CONDITIONAL(ONE, true)
-AM_CONDITIONAL(TWO, false)
-AM_CONDITIONAL(THREE, false)
+CC=false; AC_SUBST([CC])
+OBJEXT=oo; AC_SUBST([OBJEXT])
+AM_CONDITIONAL([ONE], [true])
+AM_CONDITIONAL([TWO], [false])
+AM_CONDITIONAL([THREE], [false])
AC_OUTPUT
END
targ_SOURCES = $(SONE) $(STWO) $(STHREE) $(STHREE2)
-echo:
- echo BEG: $(targ_OBJECTS) :END;
+.PHONY: test
+test:
+ test "`echo $(targ_OBJECTS)`" = "one.oo two.oo three.oo three2.oo"
END
$ACLOCAL
$AUTOCONF
-$AUTOMAKE
+$AUTOMAKE --ignore-deps
./configure
-OBJEXT=oo $MAKE -e echo > output
-cat output
-$FGREP 'BEG: one.oo two.oo three.oo three2.oo :END' output
+$MAKE test
+
+:
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Check that conditional redefinitions of AC_SUBST'ed variables are detected.
-# Report from Patrik Weiskircher
+# Report from Patrik Weiskircher.
. ./defs || Exit 1
$ACLOCAL
AUTOMAKE_fails
-grep 'libdir was already defined' stderr
+grep '^Makefile\.am:2:.* libdir was already defined' stderr
+
+:
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Check that conditional redefinitions of AC_SUBST'ed variables are detected.
-# Report from Patrik Weiskircher
+# Report from Patrik Weiskircher.
. ./defs || Exit 1
cat >>configure.in <<EOF
AC_SUBST([foo], [bar])
AM_CONDITIONAL([COND], [true])
-AC_OUTPUT
EOF
cat >Makefile.am <<EOF
if COND
+## A dummy comment to change line numer.
foo = baz
endif
EOF
$ACLOCAL
AUTOMAKE_fails
-grep 'foo was already defined' stderr
+grep '^Makefile\.am:3:.* foo was already defined' stderr
+grep '^configure\.in:4:.*foo.* previously defined here' stderr
+
+:
AUTOMAKE_fails
grep ' USE_FOO' stderr && Exit 1
grep '!USE_FOO' stderr
+
+:
# Check that many conditions do not lead to combinatorial explosion.
# (This is related to PR/352.)
-#
-# On this test, Automake 1.7.x would compute all 2**22 = 4194304
-# possible combinations of conditionals (it would do this five times,
-# to define a01_DEPENDENCIES, a02_DEPENDENCIES, a03_OBJECTS,
-# a04_OBJECTS, and to rewrite bin_PROGRAM), eating all memory, swap,
-# or cpu time it can found. Although this test won't print `FAIL' if
-# it fails, it will take long enough so it can't go unnoticed.
. ./defs || Exit 1
a03_SOURCES =
EOF
+# On this test, Automake 1.7.x would compute all 2**22 = 4194304
+# possible combinations of conditionals (it would do this five times,
+# to define a01_DEPENDENCIES, a02_DEPENDENCIES, a03_OBJECTS,
+# a04_OBJECTS, and to rewrite bin_PROGRAM), eating all memory, swap,
+# or cpu time it can found.
+(ulimit -v 1; sh -c ":") && skip_ "no adequate 'ulimit' builtin found"
+(ulimit -v 20000; sh -c ":") || skip_ "no adequate 'ulimit' builtin found"
+ulimit -v 20000
+
for i in 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22; do
-cat >>Makefile.am <<EOF
-if C$i
-bin_PROGRAMS += a$i
-a01_LDADD = foo${i}.o
-a02_LDADD += bar${i}.o
-a03_SOURCES += baz${i}.c
-a04_SOURCES = quux${i}.c
-endif C$i
+ unindent >>Makefile.am <<EOF
+ if C$i
+ bin_PROGRAMS += a$i
+ a01_LDADD = foo${i}.o
+ a02_LDADD += bar${i}.o
+ a03_SOURCES += baz${i}.c
+ a04_SOURCES = quux${i}.c
+ endif C$i
EOF
-echo "AM_CONDITIONAL([C$i], [:])" >>configure.in
+ echo "AM_CONDITIONAL([C$i], [:])" >>configure.in
done
$ACLOCAL
$AUTOMAKE
+
+:
. ./defs || Exit 1
-cat > configure.in << 'END'
-AC_INIT
-AM_INIT_AUTOMAKE(nonesuch, nonesuch)
+cat >> configure.in << 'END'
AC_PROG_CC
-AM_CONDITIONAL(ONE, true)
-AM_CONDITIONAL(TWO, false)
-AM_CONDITIONAL(THREE, maybe)
-AC_OUTPUT(Makefile)
+AM_CONDITIONAL([ONE], [true])
+AM_CONDITIONAL([TWO], [false])
+AM_CONDITIONAL([THREE], [maybe])
+AC_OUTPUT
END
cat > Makefile.am << 'END'
EOF
diff expected produced
+
+:
. ./defs || Exit 1
cat >>configure.in <<'EOF'
-AC_PROG_CC
-AM_CONDITIONAL(C1, [test -z "$two"])
-AM_CONDITIONAL(C2, [test -n "$two"])
+AM_CONDITIONAL([C1], [test -z "$two"])
+AM_CONDITIONAL([C2], [test -n "$two"])
AC_OUTPUT
EOF
-cat >>Makefile.am <<'EOF'
+cat > Makefile.am <<'EOF'
+AUTOMAKE_OPTIONS = no-dependencies
+CC = false
+EXEEXT = .foo
+
if C1
bin_PROGRAMS = a
endif
bin_PROGRAMS = b $(undefined)
endif
-print:
- @echo 'BEG: $(bin_PROGRAMS) :END'
+.PHONY: test-a test-b
+test-a:
+ test a.foo = $(bin_PROGRAMS)
+test-b:
+ test b.foo = $(bin_PROGRAMS)
EOF
$ACLOCAL
$AUTOCONF
$AUTOMAKE
+$FGREP 'a_SOURCES = a.c' Makefile.in
+$FGREP 'b_SOURCES = b.c' Makefile.in
+
./configure
-EXEEXT=.foo $MAKE -e print > stdout
-cat stdout
-grep 'BEG: a.foo :END' stdout
+$MAKE test-a
./configure two=yes
-EXEEXT=.foo $MAKE -e print > stdout
-cat stdout
-grep 'BEG: b.foo :END' stdout
+$MAKE test-b
-grep 'a_SOURCES = a.c' Makefile.in
-grep 'b_SOURCES = b.c' Makefile.in
+:
. ./defs || Exit 1
cat >>configure.in <<'EOF'
-AC_PROG_CC
-AM_CONDITIONAL(C1, [test -z "$two"])
-AM_CONDITIONAL(C2, [test -n "$two"])
+AC_SUBST([CC], [false])
+AC_SUBST([OBJEXT], [o])
+AM_CONDITIONAL([C1], [test -z "$two"])
+AM_CONDITIONAL([C2], [test -n "$two"])
AC_OUTPUT
EOF
-cat >>Makefile.am <<'EOF'
+cat > Makefile.am <<'EOF'
bin_PROGRAMS = a
a_LDADD = c0.o -L/some/where
if C1
if C2
a_LDADD += c2.o -dlopen c3.la
endif
-print:
- @echo BEG: $(a_DEPENDENCIES) :END
+
+.PHONY: test1 test2
+test1:
+ test "`echo $(a_DEPENDENCIES)`" = "c0.o c1.o"
+test2:
+ test "`echo $(a_DEPENDENCIES)`" = "c0.o c2.o c3.la"
EOF
$ACLOCAL
$AUTOCONF
-$AUTOMAKE
+$AUTOMAKE --ignore-deps
./configure
-$MAKE -e print > stdout
-cat stdout
-grep 'BEG: c0.o c1.o :END' stdout
+$MAKE test1
./configure two=yes
-$MAKE -e print > stdout
-cat stdout
-grep 'BEG: c0.o c2.o c3.la :END' stdout
+$MAKE test2
+
+:
. ./defs || Exit 1
cat >>configure.in <<'EOF'
-AC_PROG_CC
-AM_CONDITIONAL(C1, [test -z "$two"])
-AM_CONDITIONAL(C2, [test -n "$two"])
-AM_CONDITIONAL(C3, [test -z "$three"])
-AC_SUBST([MYSUB], [foo.o])
+AM_CONDITIONAL([C1], [test -z "$two"])
+AM_CONDITIONAL([C2], [test -n "$two"])
+AM_CONDITIONAL([C3], [test -z "$three"])
+# We define CC in Makefile.am, but OBJEXT here.
+OBJEXT=o; AC_SUBST([OBJEXT])
+AC_SUBST([MYSUB], ["foo.$OBJEXT"])
AC_OUTPUT
EOF
cat >>Makefile.am <<'EOF'
+AUTOMAKE_OPTIONS = no-dependencies
+CC = :
+
bin_PROGRAMS = a
+
if C1
a_LDADD = $(MYSUB)
a_DEPENDENCIES = $(MYSUB) nonsense.a
# Note that `nonsense.a' is there just to make sure Automake insn't
# using some self computed a_DEPENDENCIES variable.
endif
+
if C2
if C3
BAR = bar.o
endif
a_LDADD = $(BAR)
endif
-print:
- @echo BEG: $(a_DEPENDENCIES) :END
+
+got = `echo $(a_DEPENDENCIES)`
+test:
+ test "$(exp)" = "$(got)"
+.PHONY: test
EOF
$ACLOCAL
$AUTOMAKE
./configure
-$MAKE -e print > stdout
-cat stdout
-grep 'BEG: foo.o nonsense.a :END' stdout
+exp='foo.o nonsense.a' $MAKE -e test
./configure two=yes three=
-$MAKE -e print > stdout
-cat stdout
-grep 'BEG: bar.o :END' stdout
+exp='bar.o' $MAKE -e test
./configure two=yes three=yes
-$MAKE -e print > stdout
-cat stdout
-grep 'BEG: baz.o :END' stdout
+exp='baz.o' $MAKE -e test
+
+:
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Check for conditional library with a conditional directory.
-# Report from Ralf Corsepius
+# Check that conditionally-defined install directories are handled
+# correctly.
+# Report from Ralf Corsepius.
. ./defs || Exit 1
AC_OUTPUT
EOF
-cat >>Makefile.am <<'EOF'
+cat > Makefile.am <<'EOF'
if INC
include_foodir = $(includedir)/foo
include_foo_HEADERS = foo.h
foo.h x.sh:
:>$@
+
+.PHONY: distdircheck
distdircheck: distdir
test -f $(distdir)/foo.h
test -f $(distdir)/x.sh
$AUTOCONF
$AUTOMAKE
+cwd=`pwd` || fatal_ "cannot get current directory"
mkdir nowhere
chmod a-w nowhere
-./configure "--prefix=`pwd`/nowhere" "--bindir=`pwd`/bin" "--includedir=`pwd`/inc"
+
+./configure --prefix="$cwd"/nowhere --bindir="$cwd"/bin \
+ --includedir="$cwd"/inc
$MAKE installdirs
test ! -d bin
test -d inc/foo
$MAKE install
test ! -d bin
test -f inc/foo/foo.h
+$MAKE distdircheck
+
rm -rf inc
-./configure two=two \
- "--prefix=`pwd`/nowhere" "--bindir=`pwd`/bin" "--includedir=`pwd`/inc"
+
+./configure two=two --prefix="$cwd"/nowhere --bindir="$cwd"/bin \
+ --includedir="$cwd"/inc
$MAKE install
test ! -d inc
test -f bin/x.sh
+rm -rf inc
$MAKE installdirs
test ! -d inc
test -d bin
$MAKE distdircheck
+
+:
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Check for _DEPDENDENCIES definition with conditional _LDADD.
-# Report from Elena A. Vengerova
+# Check for _DEPENDENCIES definition with conditional _LDADD.
+# Report from Elena A. Vengerova.
. ./defs || Exit 1
cat >>configure.in <<'EOF'
-AM_CONDITIONAL([TWO], test -n "$two")
-AC_PROG_CC
+AM_CONDITIONAL([TWO], [test -n "$two"])
AC_OUTPUT
EOF
-cat >>Makefile.am <<'EOF'
+cat > Makefile.am <<'EOF'
OBJEXT=z
+CC=false
+AUTOMAKE_OPTIONS=no-dependencies
bin_PROGRAMS = test1 test2
test1_DEPENDENCIES = $(test1_LDADD) somethingelse.a
+.PHONY: dep-test1 dep-test2
dep-test1:
echo BEG: $(test1_DEPENDENCIES) :END
dep-test2:
echo BEG: $(test2_DEPENDENCIES) :END
-
EOF
-:> test.c
-
$ACLOCAL
$AUTOCONF
$AUTOMAKE
./configure
-$MAKE dep-test1 >out
-grep 'BEG: one.z somethingelse.a :END' out
-$MAKE dep-test2 >out
-grep 'BEG: three.z :END' out
+
+$MAKE dep-test1 >stdout || { cat stdout; Exit 1; }
+cat stdout
+$FGREP 'BEG: one.z somethingelse.a :END' stdout
+
+$MAKE dep-test2 >stdout || { cat stdout; Exit 1; }
+cat stdout
+$FGREP 'BEG: three.z :END' stdout
./configure two=2
-$MAKE dep-test1 >out
-grep 'BEG: two.z somethingelse.a :END' out
-$MAKE dep-test2 >out
-grep 'BEG: two.z somethingelse.a :END' out
+
+$MAKE dep-test1 >stdout || { cat stdout; Exit 1; }
+cat stdout
+$FGREP 'BEG: two.z somethingelse.a :END' stdout
+
+$MAKE dep-test2 >stdout || { cat stdout; Exit 1; }
+cat stdout
+$FGREP 'BEG: two.z somethingelse.a :END' stdout
+
+:
# Check rules output for parser defined conditionally.
# Report from Roman Fietze.
-required='flex yacc gcc'
+required='cc flex yacc'
. ./defs || Exit 1
cat >>configure.in <<'EOF'
-AM_CONDITIONAL([CASE_A], test -z "$case_B")
+AM_CONDITIONAL([CASE_A], [test -z "$case_B"])
AC_PROG_CC
AM_PROG_LEX
AC_PROG_YACC
AC_OUTPUT
EOF
-cat >>Makefile.am <<'EOF'
+cat > Makefile.am <<'EOF'
AM_YFLAGS = -d
BUILT_SOURCES = tparse.h
$AUTOCONF
$AUTOMAKE --add-missing
-test `grep tparse.h: Makefile.in | wc -l` = 1
+$FGREP 'tparse.h' Makefile.in # For debugging.
+test `$FGREP -c 'tparse.h:' Makefile.in` = 1
cat > tscan.l << 'END'
%%
"END" return EOF;
+%%
+/* Avoid possible link errors. */
+int yywrap (void)
+{
+ return 1;
+}
END
cat > tparse.y << 'END'
foobar : 'f' 'o' 'o' 'b' 'a' 'r' EOF {};
END
-cat >ta.c <<'END'
-int main()
+cat > ta.c << 'END'
+int main (void)
{
return 0;
}
./configure case_B=yes
$MAKE
$MAKE test-tb
+
+:
. ./defs || Exit 1
cat >>configure.in <<'EOF'
-AM_CONDITIONAL([CASE_A], test -z "$case_B")
+AM_CONDITIONAL([CASE_A], [test -z "$case_B"])
AC_PROG_CC
AM_PROG_LEX
AC_PROG_YACC
AC_OUTPUT
EOF
-cat >>Makefile.am <<'EOF'
+cat > Makefile.am <<'EOF'
AM_YFLAGS = -d
BUILT_SOURCES = tparse.h
# Presently Automake doesn't fully support partially overriden rules
# and should complain.
AUTOMAKE_fails --add-missing
-grep 'tparse.h.*already defined' stderr
+grep 'tparse\.h.*already defined' stderr
$AUTOMAKE -Wno-error
# Still and all, it should generate two rules.
-test `grep tparse.h: Makefile.in | wc -l` = 2
-grep '@CASE_A_TRUE@tparse.h:' Makefile.in
-grep '@CASE_A_FALSE@tparse.h:' Makefile.in
+$FGREP 'tparse.h' Makefile.in # For debugging.
+test `$FGREP -c 'tparse.h:' Makefile.in` = 2
+$FGREP '@CASE_A_TRUE@tparse.h:' Makefile.in
+$FGREP '@CASE_A_FALSE@tparse.h:' Makefile.in
:
. ./defs || Exit 1
cat >>configure.in <<'EOF'
-AM_CONDITIONAL([CASE_A], test -n "$case_A")
+AM_CONDITIONAL([CASE_A], [test -n "$case_A"])
AC_OUTPUT
EOF
-cat >>Makefile.am <<'EOF'
+cat > Makefile.am <<'EOF'
if CASE_A
check-local:
@echo GrepMe1
$AUTOMAKE
./configure
-$MAKE check >stdout
+$MAKE check >stdout || { cat stdout; Exit 1; }
cat stdout
grep GrepMe1 stdout && Exit 1
-$MAKE install >stdout
+$MAKE install >stdout || { cat stdout; Exit 1; }
cat stdout
grep GrepMe2 stdout
./configure case_A=1
-$MAKE check >stdout
+$MAKE check >stdout || { cat stdout; Exit 1; }
cat stdout
grep GrepMe1 stdout
-$MAKE install >stdout
+$MAKE install >stdout || { cat stdout; Exit 1; }
cat stdout
grep GrepMe2 stdout && Exit 1
AC_OUTPUT
EOF
-cat >>Makefile.am <<'EOF'
+cat > Makefile.am <<'EOF'
SUBDIRS = a
if CASE_A
SUBDIRS += b
SUBDIRS += iXYZ
SUBDIRS += jZYX
endif
-print:
- @echo BEG: $(SUBDIRS) :END
+
+.PHONY: test
+test:
+ test "`echo $(SUBDIRS)`" = 'a b c d e f g h iXYZ jZYX'
EOF
mkdir a b c d e f g h iXYZ jZYX
$AUTOMAKE
./configure
-$MAKE print >stdout
-cat stdout
-# Check good ordering
-grep 'BEG: a b c d e f g h iXYZ jZYX :END' stdout
# Make sure no extra variable was created for the last 3 items.
grep 'append.*=.* h iXYZ jZYX' Makefile
+# Check good ordering.
+$MAKE test
+
+:
# Build either as CONFIG_FILE or as PROGRAM.
+required=cc
. ./defs || Exit 1
+
mkdir sub
cat >>configure.in <<'END'
# Another sources-in-conditional test. Report from Tim Goodwin.
-required='GNUmake gcc'
. ./defs || Exit 1
cat >> configure.in << 'END'
-AC_PROG_CC
+AC_SUBST([CC], [false])
+AC_SUBST([OBJEXT], [o])
AM_CONDITIONAL([ONE], [test "x$CONDITION1" = "xtrue"])
AM_CONDITIONAL([TWO], [test "x$CONDITION2" = "xtrue"])
AC_OUTPUT
targ_SOURCES = main.c $(OPT1) $(OPT2)
-echo-objects:
- @echo $(targ_OBJECTS)
+got = `echo $(targ_OBJECTS)`
+
+.PHONY: test
+test:
+ test x"$(exp)" = x"$(got)"
END
$ACLOCAL
-$AUTOMAKE
+$AUTOMAKE -i
# We should not output useless definitions.
-test "`grep '^@TWO_FALSE@' Makefile.in | wc -l`" -eq 0
+grep '^@ONE_FALSE@' Makefile.in && Exit 1
+grep '^@TWO_FALSE@' Makefile.in && Exit 1
$AUTOCONF
-# Ignore user CFLAGS.
-CFLAGS=
-export CFLAGS
-
CONDITION1=true CONDITION2=true ./configure
-msgtt=`$MAKE --no-print-directory echo-objects`
+exp='main.o one.o two.o' $MAKE -e test
CONDITION1=true CONDITION2=false ./configure
-msgtf=`$MAKE --no-print-directory echo-objects`
+exp='main.o one.o' $MAKE -e test
CONDITION1=false CONDITION2=true ./configure
-msgft=`$MAKE --no-print-directory echo-objects`
+exp='main.o two.o' $MAKE -e test
CONDITION1=false CONDITION2=false ./configure
-msgff=`$MAKE --no-print-directory echo-objects`
-
-: msgtt = $msgtt
-: msgtf = $msgtf
-: msgft = $msgft
-: msgff = $msgff
-
-test x"$msgtt" = x"main.o one.o two.o"
-test x"$msgtf" = x"main.o one.o"
-test x"$msgft" = x"main.o two.o"
-test x"$msgff" = x"main.o"
+exp='main.o' $MAKE -e test
:
# Test AM_COND_IF.
. ./defs || Exit 1
+
cat >>configure.in <<'END'
AC_DEFUN([FOO],
[AC_CONFIG_FILES([$1])])
test ! -f file1
test -f file2
test ! -f file3
+
:
# AM_COND_IF with an undefined condition should fail.
. ./defs || Exit 1
+
cat >>configure.in <<'END'
-AM_COND_IF([COND],
+AM_COND_IF([BAD_COND],
[AC_CONFIG_FILES([file1])])
AC_OUTPUT
END
-$ACLOCAL && Exit 1
+$ACLOCAL 2>stderr && { cat stderr >&2; Exit 1; }
+cat stderr >&2
+grep '^configure\.in:4:.*AM_COND_IF.* no such condition.*BAD_COND' stderr
+
:
# but better to be safe.
. ./defs || Exit 1
+
cat >>configure.in <<'END'
AM_CONDITIONAL([COND], [:])
-# next line needed so that cond-if.m4 is pulled in.
+# The next line is needed so that cond-if.m4 is pulled in.
AM_COND_IF([COND])
_AM_COND_IF([COND])
AC_OUTPUT
END
+
+edit_configure_in ()
+{
+ sed "$@" < configure.in >configure.int
+ mv -f configure.int configure.in
+ rm -rf autom4te*.cache
+}
+
: >Makefile.am
$ACLOCAL
AUTOMAKE_fails
-grep 'condition stack' stderr
+grep '^configure\.in:8:.* condition stack' stderr
-sed 's/_AM_COND_IF/_AM_COND_ELSE/' < configure.in >configure.int
-mv -f configure.int configure.in
-rm -rf autom4te*.cache
+edit_configure_in 's/_AM_COND_IF/_AM_COND_ELSE/'
AUTOMAKE_fails
-grep 'else without if' stderr
+grep '^configure\.in:7:.* else without if' stderr
-sed 's/_AM_COND_ELSE/_AM_COND_ENDIF/' < configure.in >configure.int
-mv -f configure.int configure.in
-rm -rf autom4te*.cache
+edit_configure_in 's/_AM_COND_ELSE/_AM_COND_ENDIF/'
AUTOMAKE_fails
-grep 'endif without if' stderr
+grep '^configure\.in:7:.* endif without if' stderr
-sed 's/\(_AM_COND_ENDIF\).*/_AM_COND_IF\
-_AM_COND_ENDIF/' < configure.in >configure.int
-mv -f configure.int configure.in
-rm -rf autom4te*.cache
+edit_configure_in 's/\(_AM_COND_ENDIF\).*/_AM_COND_IF\
+_AM_COND_ENDIF/'
AUTOMAKE_fails
+grep '^configure\.in:7:.* not enough arguments.* _AM_COND_IF' stderr
+grep '^configure\.in:8:.* not enough arguments.* _AM_COND_ENDIF' stderr
test 2 = `grep -c 'not enough arguments' stderr`
+
:
# Ensure an error with underquoted usage of AM_COND_IF in configure.ac.
. ./defs || Exit 1
+
cat >>configure.in <<'END'
AM_CONDITIONAL([COND1], [:])
AM_CONDITIONAL([COND2], [:])
)
AC_OUTPUT
END
+
: >Makefile.am
$ACLOCAL
AUTOMAKE_fails
+$EGREP '^configure\.in:7:.* missing m4 quoting.*macro depth 2( |$)' stderr
sed '/.AM_COND_IF/{
s/^/[/
mv -f configure.int configure.in
rm -rf autom4te*.cache
$AUTOMAKE
+
+:
EOF
$ACLOCAL
-AUTOMAKE_run 0
+AUTOMAKE_run
grep 'libdir was already defined' stderr && Exit 1
grep '^libdir = ' Makefile.in && Exit 1
Exit 0
EOF
$ACLOCAL
-AUTOMAKE_run 0
+AUTOMAKE_run
grep 'foo was already defined' stderr && Exit 1
grep '^foo =' Makefile.in && Exit 1
Exit 0
. ./defs || Exit 1
cat >> configure.in << 'END'
-AM_CONDITIONAL([USE_A],[test x = y])
-AM_CONDITIONAL([USE_B],[test x = z])
+AM_CONDITIONAL([USE_A], [test x = y])
+AM_CONDITIONAL([USE_B], [test x = z])
AC_OUTPUT
END
+notcompat="incompatible with current conditional"
+
$ACLOCAL
cat > Makefile.am << 'END'
END
AUTOMAKE_fails
-grep 'endif.*incompatible with current conditional' stderr
+grep "^Makefile\\.am:2:.*endif.*!USE_A.*$notcompat.*[^!]USE_A" stderr
cat > Makefile.am << 'END'
if USE_A
END
AUTOMAKE_fails
-grep 'endif.*incompatible with current conditional' stderr
+grep "^Makefile\\.am:2:.*endif.*[^!]USE_B.*$notcompat.*[^!]USE_A" stderr
cat > Makefile.am << 'END'
if USE_A
END
AUTOMAKE_fails
-grep 'endif.*incompatible with current conditional' stderr
+grep "^Makefile\\.am:3:.*endif.*[^!]USE_A.*$notcompat.*USE_A" stderr
cat > Makefile.am << 'END'
if USE_A
END
AUTOMAKE_fails
-grep 'endif.*incompatible with current conditional' stderr
+grep "^Makefile\\.am:4:.*endif.*!USE_A.*$notcompat.*USE_B" stderr
cat > Makefile.am << 'END'
if USE_A
END
AUTOMAKE_fails
-grep 'else.*incompatible with current conditional' stderr
+grep "^Makefile\\.am:2:.*else.*[^!]USE_A.*$notcompat.*[^!]USE_A" stderr
cat > Makefile.am << 'END'
if USE_A
-else USE_B
+else !USE_A
endif
END
-AUTOMAKE_fails
-grep 'else.*incompatible with current conditional' stderr
+$AUTOMAKE
cat > Makefile.am << 'END'
if USE_A
-if USE_B
-else USE_A
-endif
+else USE_B
endif
END
AUTOMAKE_fails
-grep 'else.*incompatible with current conditional' stderr
+grep "^Makefile\\.am:2:.*else.*[^!]USE_B.*$notcompat.*[^!]USE_A" stderr
cat > Makefile.am << 'END'
if USE_A
-else !USE_A
+if USE_B
+else USE_A
+endif
endif
END
-$AUTOMAKE
+AUTOMAKE_fails
+grep "^Makefile\\.am:3:.*else.*[^!]USE_A.*$notcompat.*[^!]USE_B" stderr
+
+:
. ./defs || Exit 1
-echo 'AM_CONDITIONAL(FOO, true)' >> configure.in
+echo 'AM_CONDITIONAL([FOO], [true])' >> configure.in
cat > Makefile.am << 'END'
-
if FOO
helpdir = $(prefix)/Help
help_DATA = foo
endif
+
+a b c d e f g h:
+ touch $@
+
+.PHONY: print-data
+print-data:
+ echo BEG: $(help_DATA) :END
END
+
+# Older versions of this test checked that automake could process the above
+# Makefile.am even with no AC_OUTPUT in configure. So continue to do this
+# check, for completeness.
$ACLOCAL
$AUTOMAKE
+
+rm -rf autom4te*.cache
+
+echo AC_OUTPUT >> configure.in
+$AUTOCONF
+$AUTOMAKE Makefile
+
+./configure --prefix="`pwd`/_inst"
+
+$MAKE print-data >stdout || { cat stdout; Exit 1; }
+cat stdout
+grep '^BEG: a b c d e f g h :END$' stdout
+
+$MAKE install
+for x in a b c d e f g h; do
+ test -f _inst/Help/$x
+done
+
+:
cat >> configure.in << 'END'
AC_PROG_CC
-AM_CONDITIONAL(X, false)
+AM_CONDITIONAL([X], [test "$x" = yes])
+AC_OUTPUT
END
cat > Makefile.am << 'END'
else
noinst_PROGRAMS = x y
endif
+
+.PHONY: get-built get-install not-install
+get-built:
+ test -f x.$(OBJEXT)
+ test -f y.$(OBJEXT)
+ test -f x$(EXEEXT)
+ test -f y$(EXEEXT)
+get-installed:
+ test -f $(bindir)/x$(EXEEXT)
+ test -f $(bindir)/y$(EXEEXT)
+not-installed:
+ if find $(prefix) -type f | grep .; then exit 1; else :; fi
END
$ACLOCAL
$AUTOMAKE
+$AUTOCONF
+
+cat > x.c <<'END'
+int main (void)
+{
+ return 0;
+}
+END
+
+cp x.c y.c
+
+instdir=`pwd`/_inst || fatal_ "cannot get current directory"
+
+# Skip the rest of the test in case of e.g. missing C compiler.
+./configure --prefix="$instdir" x=yes || Exit $?
+$MAKE install
+$MAKE get-built
+$MAKE get-installed
+
+$MAKE distclean
+rm -rf _inst
+
+./configure --prefix="$instdir" x=no
+$MAKE install
+$MAKE get-built
+$MAKE not-installed
+
+:
. ./defs || Exit 1
cat >> configure.in << 'END'
-AM_CONDITIONAL(WRONG, [test x = y])
+AM_CONDITIONAL([WRONG], [test x = y])
AC_OUTPUT
END
this=is_something_interesting
endif
-echo-something:
- echo '$(this)'
+.PHONY: test-this
+test-this:
+ test '$(this)' = is_something_interesting
END
$ACLOCAL
$AUTOCONF
$AUTOMAKE -a
./configure
-$MAKE echo-something | grep interesting > /dev/null
+$MAKE test-this
+
+:
. ./defs || Exit 1
cat >> configure.in << 'END'
-AC_PROG_CC
dnl Define a macro with the same name as the conditional to exhibit
dnl any underquoted bug.
AC_DEFUN([COND1], ["some'meaningless;characters`])
-AM_CONDITIONAL([COND1], false)
+AM_CONDITIONAL([COND1], [false])
AC_CONFIG_FILES([foo/Makefile])
AC_CONFIG_FILES([bar/Makefile])
AC_OUTPUT
END
cat > Makefile.am << 'END'
+AUTOMAKE_OPTIONS = no-dependencies
+CC = false
+
SUBDIRS = foo
if COND1
SUBDIRS += bar
endif
-# Small example from the manual
+# Small example from the manual.
bin_PROGRAMS = hello
hello_SOURCES = hello-common.c
if COND1
hello_SOURCES += hello-generic.c
endif
+.PHONY: test
test: distdir
test -f $(distdir)/foo/Makefile.am
test -f $(distdir)/bar/Makefile.am
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Test for bug with conditional library.
-# From Harlan Stenn
+# From Harlan Stenn.
. ./defs || Exit 1
cat > Makefile.am << 'END'
if COND
man_MANS = foo.1
+man4_MANS = 6.man
else
-man_MANS = bar.2
+man_MANS = bar.2 baz.1
+man5_MANS = zap.5
endif
.PHONY: test1 test2
test1:
+ find $(mandir) ;: For debugging.
test -f $(mandir)/man1/foo.1
+ test -f $(mandir)/man4/6.4
test ! -f $(mandir)/man2/bar.2
+ test ! -f $(mandir)/man1/baz.1
+ test ! -f $(mandir)/man5/zap.5
test2:
- test ! -f $(mandir)/man1/foo.1
+ find $(mandir) ;: For debugging.
test -f $(mandir)/man2/bar.2
+ test -f $(mandir)/man1/baz.1
+ test -f $(mandir)/man5/zap.5
+ test ! -f $(mandir)/man1/foo.1
+ test ! -f $(mandir)/man4/6.4
+ test ! -f $(mandir)/man4/6.man
END
$ACLOCAL
$AUTOMAKE
$AUTOCONF
-mkdir dir1
-cd dir1
+$EGREP 'MANS|\.([123456789]|man)' Makefile.in # For debugging.
+
+mkdir build
+cd build
../configure FOO=true --prefix="`pwd`/_inst"
-sed -n '/man_MANS/p' Makefile # useful for debugging
-: > foo.1
+$EGREP 'MANS|\.([123456789]|man)' Makefile # For debugging.
+touch foo.1 6.man
$MAKE install
$MAKE test1
cd ..
-mkdir dir2
-cd dir2
-../configure FOO=false --prefix="`pwd`/_inst"
-sed -n '/man_MANS/p' Makefile # useful for debugging
-: > bar.2
+./configure FOO=false --prefix="`pwd`/_inst"
+$EGREP 'MANS|\.([123456789]|man)' Makefile # For debugging.
+touch bar.2 baz.1 zap.5
$MAKE install
$MAKE test2
# Make sure rebuild rules work even when AC_CONFIG_FILES uses colons.
# Report from Alexander Turbov.
-required=GNUmake
. ./defs || Exit 1
cat >> configure.in << 'END'
$MAKE
cd bar
-$MAKE top-rule >stdout
+$MAKE top-rule >stdout || { cat stdout; Exit 1; }
cat stdout
grep 'top rule' stdout
-$MAKE bot-rule >stdout
+$MAKE bot-rule >stdout || { cat stdout; Exit 1; }
cat stdout
grep 'bot rule' stdout
cd ..
$MAKE
cd bar
-$MAKE top-rule >stdout
+$MAKE top-rule >stdout || { cat stdout; Exit 1; }
cat stdout
grep 'top2 rule' stdout
-$MAKE bot-rule >stdout
+$MAKE bot-rule >stdout || { cat stdout; Exit 1; }
cat stdout
grep 'bot rule' stdout
cd ..
$MAKE
cd bar
-$MAKE top-rule >stdout
+$MAKE top-rule >stdout || { cat stdout; Exit 1; }
cat stdout
grep 'top2 rule' stdout
-$MAKE bot-rule >stdout
+$MAKE bot-rule >stdout || { cat stdout; Exit 1; }
cat stdout
grep 'bot2 rule' stdout
cd ..
$ACLOCAL
$AUTOCONF
+$AUTOHEADER
$AUTOMAKE
./configure
test: distdir
test -f $(distdir)/config.h.in
test -f $(distdir)/include/config.h.in.in
- : # Solaris Sh does not support 'test -e'
+ : # Solaris Sh does not support 'test -e'.
test ! -f $(distdir)/include/config.h.in
test ! -r $(distdir)/include/config.h.in
END
$sleep
echo '#undef @FOO@' > include/config.h.in.in
$MAKE include/config.h
-# Don't try to be too strict in this grepping, since the substutition
+# Don't try to be too strict in this grepping, since the substitution
# is done by config.status, and we don't have too much control on it.
grep '/\*.*#undef.*NameToBeGrepped' include/config.h
$MAKE test
# idea is that if config.h is in a subdir, and there is no Makefile in
# that subdir, then we want to build config.h as the top level.
-required=GNUmake
. ./defs || Exit 1
cat >> configure.in << 'END'
grep 'configure\.ac.*configure\.in.*both present' stderr
grep 'proceeding.*configure\.ac' stderr
-AUTOMAKE_run 0 -Wno-error
+AUTOMAKE_run -Wno-error
grep 'configure\.ac.*configure\.in.*both present' stderr
grep 'proceeding.*configure\.ac' stderr
test ! -r $(distdir)/dest3
test ! -r $(distdir)/dest2
test -f $(distdir)/src2
-## src3 cannot be distributed, Automake knows nothing about it
+## src3 cannot be distributed, Automake knows nothing about it.
test ! -r $(distdir)/sdir/src3
test ! -r $(distdir)/src3
END
my_dest=dest
AC_CONFIG_LINKS([sdir/dest2:src2 sdir/dest3:$my_src_dir/src3])
AC_CONFIG_LINKS([$my_dest:src])
-# the following is a link whose source is itself a link
+# The following is a link whose source is itself a link.
AC_CONFIG_LINKS([dest4:sdir/dest2])
# Some package prefer to compute links.
cmplink='dest5:src';
test ! -r sdir/dest2
test ! -r sdir/dest3
test -r dest # Should still exist, Automake knows nothing about it.
-test -r dest5 # ditto
+test -r dest5 # Ditto.
rm -f dest dest5
test ! -r dest4
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Test to make sure `-c' works. Report from Andris Pavenis.
+# See also the much more in-depth test `add-missing'.
. ./defs || Exit 1
+# First a simple test, where the auxdir is automatically determined
+# by automake.
+
: > Makefile.am
rm -f install-sh
$ACLOCAL
$AUTOMAKE -c -a
+ls -l # For debugging.
test -f install-sh
test ! -h install-sh
+# Let's do a couple of more elaborated tests, this time with the auxdir
+# explicitly defined in configure.in.
+
+mkdir sub
+cd sub
+
+cat > configure.in <<END
+AC_INIT([$me], [1.0])
+AC_CONFIG_AUX_DIR([auxdir])
+AM_INIT_AUTOMAKE
+AC_PROG_CC
+AC_CONFIG_FILES([Makefile])
+AC_OUTPUT
+END
+
+cat > Makefile.am <<END
+bin_PROGRAMS = foo
+END
+
+$ACLOCAL
+
+# `automake -a' called without `-c' should create symlinks by default,
+# even when there is already a non-symlinked required auxiliary file.
+
+mkdir auxdir
+echo FAKE-DEPCOMP > auxdir/depcomp
+$AUTOMAKE -a
+ls -l auxdir # For debugging.
+test -f auxdir/install-sh
+test -h auxdir/install-sh
+test -f auxdir/depcomp
+test ! -h auxdir/depcomp
+test FAKE-DEPCOMP = `cat auxdir/depcomp`
+
+# `automake -a -c' should not create symlinks, even when there are
+# already symlinked required auxiliary files.
+
+rm -rf auxdir
+mkdir auxdir
+cd auxdir
+ln -s "$top_testsrcdir/lib/missing" "$top_testsrcdir/lib/install-sh" .
+cd ..
+
+$AUTOMAKE -a -c
+ls -l auxdir # For debugging.
+test -f auxdir/install-sh
+test -h auxdir/install-sh
+test -f auxdir/missing
+test -h auxdir/missing
+test -f auxdir/depcomp
+test ! -h auxdir/depcomp
+diff $top_testsrcdir/lib/depcomp auxdir/depcomp
+
:
$MAKE check
test -f check-target-has-run
test ! -r all-target-has-failed
-# sanity checks
+# Sanity checks.
$MAKE && Exit 1
test -f all-target-has-failed
# Check that cygnus mode disables automatic dependency tracking.
# And check that this *cannot* be overridden.
+required=cc
. ./defs || Exit 1
cat >> configure.in <<'END'
# Try again, this time enabling cygnus mode from configure.in.
cp configure.in configure.sav
sed 's/^AM_INIT_AUTOMAKE$/&([gnits cygnus])/' configure.sav >configure.in
-cmp configure.in configure.sav && Exit 99 # sanity check
+cmp configure.in configure.sav && fatal_ 'failed to edit configure.in'
+
$ACLOCAL --force
$AUTOMAKE -Werror
mv -f configure.sav configure.in
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
+########################################################
+### IMPORTANT NOTE: keep this file 'set -e' clean. ###
+########################################################
+
+# NOTE: This file should execute correctly with any system's /bin/sh
+# shell, and not only with configure-time detected $CONFIG_SHELL,
+# *until differently and explicitly specified*.
+
## -------------------------------------------------------- ##
## Source static setup and definitions for the testsuite. ##
## -------------------------------------------------------- ##
-# This code needs to be 'set -e' clean.
-
# Ensure we are running from the right directory.
test -f ./defs-static || {
echo "$0: ./defs-static: not found in current directory" >&2
# Source the shell sanitization and variables' definitions.
. ./defs-static || exit 99
+# Enable the errexit shell flag early.
+set -e
+
# The name of the current test (without the `.test' suffix).
# Test scripts can override it if they need to (but this should
# be done carefully, and *before* including ./defs).
if test -z "$me"; then
# Guard against failure to spawn sed (seen on MSYS), or empty $argv0.
- me=`echo "$argv0" | sed -e 's,.*[\\/],,;s/\.test$//'` \
+ me=`echo "$argv0" | sed -e 's,.*[\\/],,;s/\.test$//;s/\.tap$//;'` \
&& test -n "$me" \
|| { echo "$argv0: failed to define \$me" >&2; exit 99; }
fi
-## ---------------------------------------- ##
-## Sanity checks and environment cleanup. ##
-## ---------------------------------------- ##
+## ---------------------- ##
+## Early sanity checks. ##
+## ---------------------- ##
# A single whitespace character.
sp=' '
exit 99
}
+
+## ------------------------------------ ##
+## Ensure we run with a proper shell. ##
+## ------------------------------------ ##
+
+# Make sure we run with the shell detected at configure time (unless
+# the user forbids it).
+case ${AM_TESTS_REEXEC-yes} in
+ n|no|false|0)
+ ;;
+ *)
+ # Ensure we can find ourselves.
+ if test ! -f "$0"; then
+ echo "$me: unable to find myself: $0" >&2
+ exit 99
+ fi
+ AM_TESTS_REEXEC=no; export AM_TESTS_REEXEC
+ # Cannot simply do `opts=$-', since the content of $- is not
+ # portable among different shells. So try to propagate only
+ # the portable and interesting options.
+ case $- in
+ *x*v*|*v*x) opts=-vx;;
+ *v*) opts=-v;;
+ *x*) opts=-x;;
+ *) opts=;;
+ esac
+ echo $me: exec $SHELL $opts "$0" "$*"
+ exec $SHELL $opts "$0" ${1+"$@"} || {
+ echo "$me: failed to re-execute with $SHELL" >&2
+ exit 99
+ }
+ ;;
+esac
+
+# NOTE: From this point on, we can assume this file is being executed
+# by the configure-time detected $CONFIG_SHELL.
+
+
+## ---------------------- ##
+## Environment cleanup. ##
+## ---------------------- ##
+
+# Temporarily disable this, since some shells (e.g., older version
+# of Bash) can return a non-zero exit status upon the when a non-set
+# variable is unset.
+set +e
+
# Unset some MAKE... variables that may cause $MAKE to act like a
# recursively invoked sub-make. Any $MAKE invocation in a test is
# conceptually an independent invocation, not part of the main
unset RECHECK_LOGS
unset VERBOSE
+# Re-enable, it had been temporarily disabled above.
+set -e
## ---------------------------- ##
## Auxiliary shell functions. ##
## ---------------------------- ##
+# This is used in `Exit' and in the exit trap. See comments in the latter
+# for more information,
+am__test_skipped=no
+
# We use a trap below for cleanup. This requires us to go through
# hoops to get the right exit status transported through the signal.
# So use `Exit STATUS' instead of `exit STATUS' inside of the tests.
Exit ()
{
set +e
- (exit $1)
- exit $1
+ # See comments in the exit trap for the reason we do this.
+ test 77 = $1 && am__test_skipped=yes
+ (exit $1); exit $1
}
-# Print warnings (e.g., about skipped and failed tests) to this file
-# number. Override by putting, say:
-# stderr_fileno_=9; export stderr_fileno_; exec 9>&2;
-# in the definition of AM_TESTS_ENVIRONMENT.
-# This is useful when using automake's parallel tests mode, to print the
-# reason for skip/failure to console, rather than to the *.log files.
-: ${stderr_fileno_=2}
-
-# Copied from Gnulib's `tests/init.sh'.
-warn_ () { echo "$@" 1>&$stderr_fileno_; }
-fail_ () { warn_ "$me: failed test: $@"; Exit 1; }
-skip_ () { warn_ "$me: skipped test: $@"; Exit 77; }
-fatal_ () { warn_ "$me: hard error: $@"; Exit 99; }
-framework_failure_ () { warn_ "$me: set-up failure: $@"; Exit 99; }
+if test $using_tap = yes; then
+ funcs_file_=tap-functions.sh
+else
+ funcs_file_=plain-functions.sh
+fi
+
+if test -f "$testsrcdir/$funcs_file_"; then
+ . "$testsrcdir/$funcs_file_" || {
+ echo "$me: error sourcing $testsrcdir/$funcs_file_" >&2
+ Exit 99
+ }
+else
+ echo "$me: $testsrcdir/$funcs_file_ not found, check \$testsrcdir" >&2
+ Exit 99
+fi
+unset funcs_file_
+
+# Avoid spurious TAP diagnostic. Keep this in sync with the definition
+# of $(TAP_LOG_DRIVER_FLAGS) in `tests/Makefile.am'.
+diag_string_="#%#"
# cross_compiling
# ---------------
# tests (or portions of them) that requires a native compiler.
cross_compiling ()
{
- test x"$host_alias" != x
+ test x"$host_alias" != x && test x"$build_alias" != x"$host_alias"
}
# is_newest FILE FILES
test -z "$is_newest_files"
}
-# AUTOMAKE_run status [options...]
-# --------------------------------
-# Run Automake with OPTIONS, and fail if automake
-# does not exit with STATUS.
+# AUTOMAKE_run [-e STATUS] [-d DESCRIPTION] [--] [AUTOMAKE-ARGS...]
+# -----------------------------------------------------------------
+# Run automake with AUTOMAKE-ARGS, and fail if it doesn't exit with
+# STATUS. Should be polymorphic for TAP and "plain" tests. The
+# DESCRIPTION, when provided, is used for console reporting, only if
+# the TAP protocol is in use in the current test script.
AUTOMAKE_run ()
{
- expected_exitcode=$1
- shift
- exitcode=0
- $AUTOMAKE ${1+"$@"} >stdout 2>stderr || exitcode=$?
+ am__desc=
+ am__exp_rc=0
+ while test $# -gt 0; do
+ case $1 in
+ -d) am__desc=$2; shift;;
+ -e) am__exp_rc=$2; shift;;
+ --) shift; break;;
+ # Don't fail on unknown option: assume they (and the rest of the
+ # command line) are to be passed verbatim to automake (so stop our
+ # own option parsing).
+ *) break;;
+ esac
+ shift
+ done
+ am__got_rc=0
+ $AUTOMAKE ${1+"$@"} >stdout 2>stderr || am__got_rc=$?
cat stderr >&2
cat stdout
- test $exitcode = $expected_exitcode || Exit 1
+ if test $using_tap != yes; then
+ test $am__got_rc -eq $am__exp_rc || Exit 1
+ return
+ fi
+ if test -z "$am__desc"; then
+ if test $am__got_rc -eq $am__exp_rc; then
+ am__desc="automake exited $am__got_rc"
+ else
+ am__desc="automake exited $am__got_rc, expecting $am__exp_rc"
+ fi
+ fi
+ command_ok_ "$am__desc" test $am__got_rc -eq $am__exp_rc
}
-# AUTOMAKE_fails [options...]
-# ---------------------------
-# Run Automake with OPTIONS, and fail if automake
-# does not exit with STATUS.
+# AUTOMAKE_fails [-d DESCRIPTION] [OPTIONS...]
+# --------------------------------------------
+# Run automake with OPTIONS, and fail if doesn't exit with status `1'.
+# Should be polymorphic for TAP and "plain" tests. The DESCRIPTION,
+# when provided, is used for console reporting, only if the TAP
+# protocol is in use in the current test script.
AUTOMAKE_fails ()
{
- AUTOMAKE_run 1 ${1+"$@"}
+ AUTOMAKE_run -e 1 ${1+"$@"}
}
# using_gmake
return 1
fi;;
*)
- echo "invalid value for \$am__using_gmake: '$am__using_gmake'" >&2
- Exit 99;;
+ fatal_ "invalid value for \$am__using_gmake: '$am__using_gmake'";;
esac
}
+am__using_gmake="" # Avoid interferences from the environment.
# make_can_chain_suffix_rules
# ---------------------------
}
am__can_chain_suffix_rules="" # Avoid interferences from the environment.
+# seq_ - print a sequence of numbers
+# ----------------------------------
+# This function simulates GNU seq(1) portably. Valid usages:
+# - seq LAST
+# - seq FIRST LAST
+# - seq FIRST INCREMENT LAST
+seq_ ()
+{
+ case $# in
+ 0) fatal_ "seq_: missing argument";;
+ 1) seq_first=1 seq_incr=1 seq_last=$1;;
+ 2) seq_first=$1 seq_incr=1 seq_last=$2;;
+ 3) seq_first=$1 seq_incr=$2 seq_last=$3;;
+ *) fatal_ "seq_: too many arguments";;
+ esac
+ # Try to avoid forks if possible.
+ case "$BASH_VERSION" in
+ ""|[12].*)
+ : Not bash, or a too old bash version. ;;
+ *)
+ # Use eval to protect dumber shells from parsing errors.
+ eval 'for ((i = seq_first; i <= seq_last; i += seq_incr)); do
+ echo $i
+ done'
+ return 0;;
+ esac
+ # Else, use GNU seq if available.
+ seq "$@" && return 0
+ # Otherwise revert to a slower loop using expr(1).
+ i=$seq_first
+ while test $i -le $seq_last; do
+ echo $i
+ i=`expr $i + $seq_incr`
+ done
+}
+
+# rm_rf_ [FILES OR DIRECTORIES ...]
+# ---------------------------------
+# Recursively remove the given files or directory, also handling the case
+# of non-writable subdirectories.
+rm_rf_ ()
+{
+ test $# -gt 0 || return 0
+ # Ignore failures in find, we are only interested in failures of the
+ # final rm.
+ find "$@" -type d ! -perm -700 -exec chmod u+rwx {} \; || :
+ rm -rf "$@"
+}
+
+# count_test_results total=N pass=N fail=N xpass=N xfail=N skip=N error=N
+# -----------------------------------------------------------------------
+# Check that a testsuite run driven by the parallel-tests harness has
+# had the specified numbers of test results (specified by kind).
+# This function assumes that the output of "make check" or "make recheck"
+# has been saved in the `stdout' file in the current directory, and its
+# log in the `test-suite.log' file.
+count_test_results ()
+{
+ # Use a subshell so that we won't pollute the script namespace.
+ (
+ # TODO: Do proper checks on the arguments?
+ total=ERR pass=ERR fail=ERR xpass=ERR xfail=ERR skip=ERR error=ERR
+ eval "$@"
+ # For debugging.
+ $EGREP -i '(total|x?pass|x?fail|skip|error)' stdout || :
+ rc=0
+ # Avoid spurious failures with shells with "overly sensible"
+ # `errexit' shell flag, such as e.g., Solaris /bin/sh.
+ set +e
+ test `grep -c '^PASS:' stdout` -eq $pass || rc=1
+ test `grep -c '^XFAIL:' stdout` -eq $xfail || rc=1
+ test `grep -c '^SKIP:' stdout` -eq $skip || rc=1
+ test `grep -c '^FAIL:' stdout` -eq $fail || rc=1
+ test `grep -c '^XPASS:' stdout` -eq $xpass || rc=1
+ test `grep -c '^ERROR:' stdout` -eq $error || rc=1
+ grep "^# TOTAL: *$total$" stdout || rc=1
+ grep "^# PASS: *$pass$" stdout || rc=1
+ grep "^# XFAIL: *$xfail$" stdout || rc=1
+ grep "^# SKIP: *$skip$" stdout || rc=1
+ grep "^# FAIL: *$fail$" stdout || rc=1
+ grep "^# XPASS: *$xpass$" stdout || rc=1
+ grep "^# ERROR: *$error$" stdout || rc=1
+ test $rc -eq 0
+ )
+}
+
commented_sed_unindent_prog='
/^$/b # Nothing to do for empty lines.
x # Get x<indent> into pattern space.
}
sed_unindent_prog="" # Avoid interferences from the environment.
+# get_shell_script SCRIPT-NAME
+# -----------------------------
+# Fetch an Automake-provided shell script from the `lib/' directory into
+# the current directory, and, if the `$test_prefer_config_shell' variable
+# is set to "yes", modify its shebang line to use $SHELL instead of
+# /bin/sh.
+get_shell_script ()
+{
+ if test x"$test_prefer_config_shell" = x"yes"; then
+ sed "1s|#!.*|#! $SHELL|" "$top_testsrcdir/lib/$1" > "$1"
+ chmod a+x "$1"
+ else
+ cp "$top_testsrcdir/lib/$1" .
+ fi
+ sed 10q "$1" # For debugging.
+}
+
+# require_xsi SHELL
+# -----------------
+# Skip the test if the given shell fails to support common XSI constructs.
+require_xsi ()
+{
+ test $# -eq 1 || fatal_ "require_xsi needs exactly one argument"
+ echo "$me: trying some XSI constructs with $1"
+ $1 -c "$xsi_shell_code" || skip_all_ "$1 lacks XSI features"
+}
+# Shell code supposed to work only with XSI shells. Keep this in sync
+# with libtool.m4:_LT_CHECK_SHELL_FEATURES.
+xsi_shell_code='
+ _lt_dummy="a/b/c"
+ test "${_lt_dummy##*/},${_lt_dummy%/*},${_lt_dummy#??}"${_lt_dummy%"$_lt_dummy"}, \
+ = c,a/b,b/c, \
+ && eval '\''test $(( 1 + 1 )) -eq 2 \
+ && test "${#_lt_dummy}" -eq 5'\'
+
+# fetch_tap_driver
+# ----------------
+# Fetch the Automake-provided TAP driver from the `lib/' directory into
+# the current directory, and edit its shebang line so that it will be
+# run with the perl interpreter determined at configure time.
+fetch_tap_driver ()
+{
+ # TODO: we should devise a way to make the shell TAP driver tested also
+ # TODO: with /bin/sh, for better coverage.
+ case $am_tap_implementation in
+ perl)
+ sed "1s|#!.*|#! $PERL -w|" "$top_testsrcdir"/lib/tap-driver.pl ;;
+ shell)
+ sed "1s|#!.*|#! $SHELL|" "$top_testsrcdir"/lib/tap-driver.sh ;;
+ *)
+ fatal_ "invalid \$am_tap_implementation '$am_tap_implementation'" ;;
+ esac > tap-driver \
+ && chmod a+x tap-driver \
+ || framework_failure_ "couldn't fetch $am_tap_implementation TAP driver"
+ sed 10q tap-driver # For debugging.
+}
+# The shell/awk implementation of the TAP driver is still mostly dummy, so
+# use the perl implementation by default for the moment.
+am_tap_implementation=${am_tap_implementation-shell}
## ----------------------------------------------------------- ##
## Checks for required tools, and additional setups (if any) ##
# Check that each required tool is present.
case $tool in
:) ;;
- cc|c++|fortran|fortran77)
- echo "$me: dummy requirement '$tool', no check done"
+ cc)
+ test "$CC" = false && skip_all_ "no C compiler available";;
+ c++)
+ test "$CXX" = false && skip_all_ "no C++ compiler available";;
+ fortran)
+ test "$FC" = false && skip_all_ "no Fortran compiler available";;
+ fortran77)
+ test "$F77" = false && skip_all_ "no Fortran 77 compiler available";;
+ xsi-shell)
+ require_xsi "$SHELL";;
+ xsi-bin-sh)
+ require_xsi "/bin/sh";;
+ xsi-lib-shell)
+ if test x"$test_prefer_config_shell" = x"yes"; then
+ require_xsi "$SHELL"
+ else
+ require_xsi "/bin/sh"
+ fi
;;
bzip2)
# Do not use --version, bzip2 still tries to compress stdin.
echo "$me: running bzip2 --help"
- bzip2 --help || exit 77
+ bzip2 --help \
+ || skip_all_ "required program \`bzip2' not available"
;;
cl)
CC=cl
export CC
echo "$me: running $CC -?"
- $CC -? || exit 77
+ $CC -? || skip_all_ "Microsoft C compiler \`$CC' not available"
;;
cscope)
# Sun cscope is interactive without redirection.
echo "$me: running cscope --version </dev/null"
- cscope --version </dev/null || exit 77
+ cscope --version </dev/null \
+ || skip_all_ "required program \`cscope' not available"
;;
etags)
# Exuberant Ctags will create a TAGS file even
# does not have such problem.) Use -o /dev/null
# to make sure we do not pollute the tests/ directory.
echo "$me: running etags --version -o /dev/null"
- etags --version -o /dev/null || exit 77
+ etags --version -o /dev/null \
+ || skip_all_ "required program \`etags' not available"
;;
GNUmake)
- echo "$me: determine if $MAKE is GNU make"
- using_gmake || exit 77
+ for make_ in "$MAKE" gmake gnumake :; do
+ MAKE=$make_ am__using_gmake=''
+ test "$MAKE" = : && break
+ echo "$me: determine whether $MAKE is GNU make"
+ using_gmake && break
+ done
+ test "$MAKE" = : && skip_all_ "this test requires GNU make"
+ export MAKE
+ unset make_
;;
gcc)
# When gcc is required, export `CC=gcc' so that ./configure
CC=${am__tool_prefix}gcc
export CC
echo "$me: running $CC --version"
- $CC --version || exit 77
+ $CC --version || skip_all_ "GNU C compiler not available"
echo "$me: running $CC -v"
- $CC -v || exit 77
+ $CC -v || skip_all_ "botched installation for GNU C compiler"
;;
gcj)
GCJ=${am__tool_prefix}gcj
export GCJ
echo "$me: running $GCJ --version"
- $GCJ --version || exit 77
+ $GCJ --version || skip_all_ "GNU Java compiler not available"
echo "$me: running $GCJ -v"
- $GCJ -v || exit 77
+ $GCJ -v || skip_all_ "botched installation for GNU Java compiler"
;;
g++)
CXX=${am__tool_prefix}g++
export CXX
echo "$me: running $CXX --version"
- $CXX --version || exit 77
+ $CXX --version || skip_all_ "GNU C++ compiler not available"
echo "$me: running $CXX -v"
- $CXX -v || exit 77
+ $CXX -v || skip_all_ "botched installation for GNU C++ compiler"
;;
gfortran)
FC=${am__tool_prefix}gfortran
export FC
echo "$me: running $FC --version"
- $FC --version || skip_ "GNU Fortran compiler not available"
+ $FC --version || skip_all_ "GNU Fortran compiler not available"
echo "$me: running $FC -v"
- $FC -v || skip_ "botched installation for GNU Fortran compiler"
+ $FC -v || skip_all_ "botched installation for GNU Fortran compiler"
case " $required " in
*\ g77\ *) ;;
*) F77=$FC; export F77;;
F77=${am__tool_prefix}g77
export F77
echo "$me: running $F77 --version"
- $F77 --version || skip_ "GNU Fortran 77 compiler not available"
+ $F77 --version || skip_all_ "GNU Fortran 77 compiler not available"
echo "$me: running $F77 -v"
- $F77 -v || skip_ "botched installation for GNU Fortran 77 compiler"
+ $F77 -v || skip_all_ "botched installation for GNU Fortran 77 compiler"
case " $required " in
*\ gfortran\ *) ;;
*) FC=$F77; export FC;;
# it will try link *nothing* and complain it cannot find
# main(); funny). Use -help so it does not try linking anything.
echo "$me: running $CC -V -help"
- $CC -V -help || exit 77
+ $CC -V -help || skip_all_ "Intel C compiler \`$CC' not available"
;;
javac)
# The Java compiler from JDK 1.5 (and presumably earlier versions)
# telling that source files are missing. Adding also the `-help'
# option seems to solve the problem.
echo "$me: running javac -version -help"
- javac -version -help || exit 77
+ javac -version -help || skip_all_ "Sun Java compiler not available"
;;
makedepend)
echo "$me: running makedepend -f-"
- makedepend -f- || exit 77
+ makedepend -f- \
+ || skip_all_ "required program \`makedepend' not available"
;;
makeinfo-html)
- # Make sure makeinfo understands --html.
+ # Make sure we have makeinfo, and it understands `--html'.
echo "$me: running makeinfo --html --version"
- makeinfo --html --version || exit 77
+ makeinfo --html --version \
+ || skip_all_ "cannot find a makeinfo program that groks the" \
+ "\`--html' option"
;;
non-root)
# Skip this test case if the user is root.
# We try to append to a read-only file to detect this.
priv_check_temp=priv-check.$$
- touch $priv_check_temp || exit 99
- chmod a-w $priv_check_temp || exit 99
- (echo foo >> $priv_check_temp) >/dev/null 2>&1
- overwrite_status=$?
+ touch $priv_check_temp && chmod a-w $priv_check_temp \
+ || framework_failure_ "creating unwritable file $priv_check_temp"
+ # Not a useless use of subshell: lesser shells like Solaris /bin/sh
+ # can exit if a builtin fails.
+ overwrite_status=0
+ (echo foo >> $priv_check_temp) || overwrite_status=$?
rm -f $priv_check_temp
if test $overwrite_status -eq 0; then
- echo "$me: cannot drop file write permissions" >&2
- exit 77
+ skip_all_ "cannot drop file write permissions"
fi
unset priv_check_temp overwrite_status
;;
perl-threads)
if test "$WANT_NO_THREADS" = "yes"; then
- echo "$me: skip with Devel::Cover: cannot cope with threads" >&2
- exit 77
+ skip_all_ "Devel::Cover cannot cope with threads"
fi
;;
native)
- cross_compiling && skip_ "doesn't work in cross-compile mode"
+ cross_compiling && skip_all_ "doesn't work in cross-compile mode"
;;
python)
# Python doesn't support --version, it has -V
echo "$me: running python -V"
- python -V || exit 77
+ python -V || skip_all_ "python interpreter not available"
;;
ro-dir)
# Skip this test case if read-only directories aren't supported
# (e.g., under DOS.)
ro_dir_temp=ro_dir.$$
- mkdir $ro_dir_temp || exit 99
- chmod a-w $ro_dir_temp || exit 99
- (: > $ro_dir_temp/probe) >/dev/null 2>/dev/null
- create_status=$?
+ mkdir $ro_dir_temp && chmod a-w $ro_dir_temp \
+ || framework_failure_ "creating unwritable directory $ro_dir_temp"
+ # Not a useless use of subshell: lesser shells like Solaris /bin/sh
+ # can exit if a builtin fails.
+ create_status=0
+ (: > $ro_dir_temp/probe) || create_status=$?
rm -rf $ro_dir_temp
if test $create_status -eq 0; then
- echo "$me: cannot drop directory write permissions" >&2
- exit 77
+ skip_all_ "cannot drop directory write permissions"
fi
unset ro_dir_temp create_status
;;
echo "$me: running $r2h --version"
$r2h --version && break 2
done
- echo "$me: no proper rst2html program found" >&2
- exit 77
+ skip_all_ "no proper rst2html program found"
done
unset r2h
;;
# the program on the runtest command-line. This requires
# DejaGnu 1.4.3 or later.
echo "$me: running runtest SOMEPROGRAM=someprogram --version"
- runtest SOMEPROGRAM=someprogram --version || exit 77
+ runtest SOMEPROGRAM=someprogram --version \
+ || skip_all_ "DejaGnu is not available"
;;
tex)
# No all versions of Tex support `--version', so we use
# a configure check.
if test -z "$TEX"; then
- echo "$me: TeX is required, but it wasn't found by configure" >&2
- exit 77
+ skip_all_ "TeX is required, but it wasn't found by configure"
fi
;;
texi2dvi-o)
# Texi2dvi supports `-o' since Texinfo 4.1.
echo "$me: running texi2dvi -o /dev/null --version"
- texi2dvi -o /dev/null --version || exit 77
- ;;
- xsi-shell)
- # Try some XSI features.
- # Keep this in sync with libtool.m4:_LT_CHECK_SHELL_FEATURES.
- echo "$me: trying some XSI constructs"
- ( _am_dummy="a/b/c"
- test "${_am_dummy##*/},${_am_dummy%/*},${_am_dummy#??}"${_am_dummy%"$_am_dummy"}, \
- = c,a/b,b/c, \
- && eval 'test $(( 1 + 1 )) -eq 2 \
- && test "${#_am_dummy}" -eq 5' ) || exit 77
+ texi2dvi -o /dev/null --version \
+ || skip_all_ "required program \`texi2dvi' not available"
;;
- flex|lex)
+ flex)
# Since flex is required, we pick LEX for ./configure.
LEX=flex
export LEX
echo "$me: running flex --version"
- flex --version || exit 77
+ flex --version || skip_all_ "required program \`flex' not available"
+ ;;
+ lex)
+ test "$LEX" = false && skip_all_ "no Lex program available"
+ if test -z "$LEX"; then
+ # The user hasn't explicitly specified any lex program in the
+ # environment, so we try to use flex, skipping the test if it's
+ # not found.
+ LEX=flex
+ export LEX
+ echo "$me: running flex --version"
+ flex --version || skip_all_ "required program \`flex' not available"
+ fi
;;
yacc)
- if test x"$YACC" = x"no"; then
- # The user has explicitly told he doesn't want a yacc program
- # to be used.
- echo "$me: \$YACC is \"no\", skipping test" >&2
- exit 77
- elif test -z "$YACC"; then
+ test "$YACC" = false && skip_all_ "no Yacc program available"
+ if test -z "$YACC"; then
# The user hasn't explicitly specified any yacc program in the
# environment, so we try to use bison, skipping the test if it's
# not found.
YACC='bison -y'
export YACC
echo "$me: running bison --version"
- bison --version || exit 77
+ bison --version || skip_all_ "required program \`bison' not available"
fi
;;
*)
# It is not likely but possible that $tool is a special builtin,
# in which case the shell is allowed to exit after an error.
# So, please leave the subshell here.
- ( $tool --version ) || exit 77
+ ($tool --version) || skip_all_ "required program \`$tool' not available"
;;
esac
done
*\ *|*\ *)
case " $required " in
*' libtool '* | *' libtoolize '* )
- echo "$me: libtool/libtoolized cannot cope correctly" >&2
- echo "$me: with spaces in the build tree" >&2
- exit 77
- ;;
+ skip_all_ "libtool has problems with spaces in builddir name";;
esac
;;
esac
*\ * |*\ *)
case " $required " in
*' libtool '* | *' libtoolize '* | *' gettext '* )
- echo "$me: our testsuite setup cannot cope correctly with spaces" >&2
- echo "$me: in the source tree for libtool/gettext tests" >&2
- exit 77
- ;;
+ skip_all_ "spaces in srcdir name: libtool/gettext tests won't work";;
esac
;;
esac
case " $required " in
*' libtool '*|*' libtoolize '*)
if test $libtool_found != yes; then
- echo "$me: libtool/libtoolize is required, but libtool.m4 wasn't" >&2
- echo "$me: found in directories $aclocaldir $extra_includes" >&2
- exit 77
+ # Write more diagnostic to the log file than to the console.
+ echo "$me: \`libtool.m4' wasn't found in directories" \
+ "$aclocaldir $extra_includes"
+ skip_all_ "libtool m4 macros won't be found by aclocal"
fi
;;
*' gettext '*)
if test $gettext_found != yes; then
- echo "$me: gettext is required, but gettext.m4 wasn't found" >&2
- echo "$me: in directories $aclocaldir $extra_includes" >&2
- exit 77
+ # Write more diagnostic to the log file than to the console.
+ echo "$me: \`gettext.m4' wasn't found in directories" \
+ "$aclocaldir $extra_includes"
+ skip_all_ "gettext m4 macros won't be found by aclocal"
fi
;;
esac
# Test scripts are free to override this if they need to.
distdir=$me-1.0
-# The subdirectory where the current test script will run and write its
-# temporary/data files. This will be created shortly, and will be removed
-# by the cleanup trap below if the test passes. If the test doesn't pass,
-# this directory will be kept, to facilitate debugging.
-testSubDir=$me.dir
-
-test ! -d $testSubDir || {
- find $testSubDir -type d ! -perm -700 -exec chmod u+rwx {} ";"
- rm -rf $testSubDir
-}
-mkdir $testSubDir
-
-cd ./$testSubDir
-
+# Set up the exit trap.
if test "$sh_errexit_works" = yes; then
trap 'exit_status=$?
set +e
cd "$testbuilddir"
- case $exit_status,$keep_testdirs in
- 0,)
- find $testSubDir -type d ! -perm -700 -exec chmod u+rwx {} ";"
- rm -rf $testSubDir
- ;;
- esac
- test "$signal" != 0 && echo "$me: caught signal $signal"
+ if test -n "$keep_testdirs"; then
+ keep_testdirs=yes
+ else
+ keep_testdirs=no
+ fi
+ if test $using_tap = yes; then
+ if test $have_tap_plan_ != yes && test $exit_status -eq 0; then
+ plan_ "now"
+ fi
+ test $exit_status -eq 0 && test $tap_pass_count_ -eq $tap_count_ \
+ || keep_testdirs=yes
+ else
+ # This is to ensure that a test script does give a SKIP outcome just
+ # because a command in it happens to exit with status 77. This
+ # behaviour, while from time to time useful to developers, is not
+ # meant to be enabled by default, as it could cause spurious failures
+ # in the wild. Thus it will be enabled only when the variable
+ # "am_explicit_skips" is set to a "true" value.
+ case $am_explicit_skips in
+ [yY]|[yY]es|1)
+ if test $exit_status -eq 77 && test $am__test_skipped != yes; then
+ echo "$me: implicit skip turned into failure"
+ exit_status=78
+ fi;;
+ esac
+ test $exit_status -eq 0 || keep_testdirs=yes
+ fi
+ test $keep_testdirs = no && rm_rf_ $testSubDir
+ set +x
echo "$me: exit $exit_status"
exit $exit_status
' 0
for signal in 1 2 13 15; do
- trap 'signal='$signal'; { Exit 99; }' $signal
+ trap "fatal_ 'caught signal $signal'" $signal
done
- signal=0
+ unset signal
fi
-# Copy in some files we need.
-for file in install-sh missing depcomp; do
- cp "$top_testsrcdir/lib/$file" . || Exit 99
-done
-
-# Build appropriate environment in test directory. Eg create
-# configure.in, touch all necessary files, etc.
-# Don't use AC_OUTPUT, but AC_CONFIG_FILES so that appending
-# still produces a valid configure.ac. But then, tests running
-# config.status really need to append AC_OUTPUT.
-{
- echo "AC_INIT([$me], [1.0])"
- if test x"$parallel_tests" = x"yes"; then
- echo "AM_INIT_AUTOMAKE([parallel-tests])"
- else
- echo "AM_INIT_AUTOMAKE"
+# Create and populate the temporary directory, if and as required.
+if test x"$am_create_testdir" = x"no"; then
+ testSubDir=
+else
+ # The subdirectory where the current test script will run and write its
+ # temporary/data files. This will be created shortly, and will be removed
+ # by the cleanup trap below if the test passes. If the test doesn't pass,
+ # this directory will be kept, to facilitate debugging.
+ testSubDir=$me.dir
+ test ! -d $testSubDir || rm_rf_ $testSubDir \
+ || framework_failure_ "removing old test subdirectory"
+ mkdir $testSubDir \
+ || framework_failure_ "creating test subdirectory"
+ cd ./$testSubDir \
+ || framework_failure_ "cannot chdir into test subdirectory"
+ if test x"$am_create_testdir" != x"empty"; then
+ cp "$top_testsrcdir"/lib/install-sh "$top_testsrcdir"/lib/missing \
+ "$top_testsrcdir"/lib/depcomp . \
+ || framework_failure_ "fetching common files from lib/"
+ # Build appropriate environment in test directory. E.g., create
+ # configure.in, touch all necessary files, etc. Don't use AC_OUTPUT,
+ # but AC_CONFIG_FILES so that appending still produces a valid
+ # configure.in. But then, tests running config.status really need
+ # to append AC_OUTPUT.
+ {
+ echo "AC_INIT([$me], [1.0])"
+ if test x"$parallel_tests" = x"yes"; then
+ echo "AM_INIT_AUTOMAKE([parallel-tests])"
+ else
+ echo "AM_INIT_AUTOMAKE"
+ fi
+ echo "AC_CONFIG_FILES([Makefile])"
+ } >configure.in || framework_failure_ "creating configure.in skeleton"
fi
- echo "AC_CONFIG_FILES([Makefile])"
-} >configure.in
+fi
## ---------------- ##
## Ready to go... ##
## ---------------- ##
set -x
-set -e
-
pwd
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Defines and minimal setup for Automake testing environment.
-# Multiple inclusions of this file should be idempotent!
-# This code needs to be 'set -e' clean.
+
+# IMPORTANT NOTES AND REQUIREMENTS
+# - Multiple inclusions of this file should be idempotent.
+# - This code has to be 'set -e' clean.
+# - This file should execute correctly with any system's /bin/sh
+# shell, not only with configure-time detected $CONFIG_SHELL.
# Be more Bourne compatible.
# (Snippet inspired to configure's initialization in Autoconf 2.64)
# Check that the environment is properly sanitized.
# Having variables exported to the empty string is OK, since our code
# treats such variables as if they were unset.
-for var in me required parallel_tests original_AUTOMAKE original_ACLOCAL; do
+for var in \
+ me \
+ required \
+ using_tap \
+ parallel_tests \
+ am_create_testdir \
+ am_tap_implementation \
+ test_prefer_config_shell \
+ original_AUTOMAKE \
+ original_ACLOCAL \
+; do
if eval "test x\"\$$var\" != x" && env | grep "^$var=" >/dev/null; then
echo "$argv0: variable \`$var' is set in the environment:" \
"this is unsafe" >&2
done
unset var
+# See whether the current test script is expected to use TAP or not.
+# Use a sensible default, while allowing the scripts to override this
+# check.
+if test -z "$using_tap"; then
+ case $argv0 in *.tap) using_tap=yes;; *) using_tap=no;; esac
+fi
+
testsrcdir='@abs_srcdir@'
top_testsrcdir='@abs_top_srcdir@'
testbuilddir='@abs_builddir@'
APIVERSION='@APIVERSION@'
PATH_SEPARATOR='@PATH_SEPARATOR@'
+host_alias=${host_alias-'@host_alias@'}; export host_alias
+build_alias=${build_alias-'@build_alias@'}; export build_alias
+
# Make sure we override the user shell.
SHELL='@SHELL@'; export SHELL
# User can override various tools used.
# this variable.
TEX='@TEX@'
-# Whether /bin/sh has working 'set -e' with exit trap.
+# Whether $SHELL has working 'set -e' with exit trap.
sh_errexit_works='@sh_errexit_works@'
# The amount we should wait after modifying files depends on the platform.
TESTS = frob.test
END
+test x"$parallel_tests" != x"yes" || : > test-driver
+
$ACLOCAL
$AUTOMAKE
# Check that the DejaGnu rules work for a simple program and test case.
# Also check PR 488: Failure of the first of several tools tested.
+# From the original bug report:
+# ``If you use dejagnu for testing and have multiple tools (i.e.,
+# multiple entries in the DEJATOOL variable) then the success/failure
+# of "make check" is only dependent on the success/failure of the
+# tests on the final tool. Thus there may be failures in the tests on
+# previous tools, but at first glance "make check" has passed.''
required=runtest
. ./defs || Exit 1
required=runtest
. ./defs || Exit 1
-# Check whether DejaGnu supports --status
-runtest --help | grep '.*--status' || Exit 77
+runtest --help | grep '.*--status' \
+ || skip_ "dejagnu lacks support for '--status'"
cat > failtcl << 'END'
#! /bin/sh
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Test to make sure depcomp is installed and found properly
-# when required for multiple directories
+# when required for multiple directories.
. ./defs || Exit 1
$AUTOMAKE --add-missing
# There used to be a bug where this was created in the first subdir with C
-# sources in it instead of in $top_srcdir or $ac_auxdir
+# sources in it instead of in $top_srcdir or $ac_auxdir.
test -f depcomp
:
+++ /dev/null
-#! /bin/sh
-# Copyright (C) 2011 Free Software Foundation, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2, or (at your option)
-# any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program. If not, see <http://www.gnu.org/licenses/>.
-
-# hp depmode should work with GNU make in VPATH mode (bug similar to
-# depcomp9.test).
-# For automake bug#8473.
-
-# Here's the bug: hp depmode will prefix VPATH to the object file name,
-# thus the second gmake will invoke depcomp with object='../../src/foo.o',
-# causing errors such as (broken on multiple lines for clarity):
-# cpp: "", line 0: error 4066: Cannot create
-# "../../gllib/.deps/nonblocking.TPo" file for
-# "-M../../gllib/.deps/nonblocking.TPo" option.
-# (No such file or directory[errno=2])
-
-required=GNUmake
-. ./defs || Exit 1
-
-mkdir src src/sub build
-
-cat >> configure.in << 'END'
-AC_PROG_CC
-AM_PROG_CC_C_O
-AC_CONFIG_FILES([src/Makefile])
-AC_OUTPUT
-END
-
-cat > Makefile.am << 'END'
-SUBDIRS = src
-END
-
-cat > src/Makefile.am << 'END'
-AUTOMAKE_OPTIONS = subdir-objects
-bin_PROGRAMS = foo
-foo_SOURCES = foo.c foo.h sub/subfoo.c
-END
-
-cat > src/foo.h <<EOF
-extern int subfoo (void);
-EOF
-
-cat >src/foo.c <<EOF
-#include "foo.h"
-int main (void)
-{
- return subfoo ();
-}
-EOF
-
-cat >src/sub/subfoo.c <<EOF
-#include "foo.h"
-int subfoo (void)
-{
- return 0;
-}
-EOF
-
-$ACLOCAL
-$AUTOCONF
-$AUTOMAKE -a
-
-cd build
-../configure am_cv_CC_dependencies_compiler_type=hp
-# Do not error out with the first make, as the forced 'hp' depmode might
-# not actually work, but we have overridden the _AM_DEPENDENCIES tests.
-$MAKE || Exit 77
-
-# We must clean and rebuild, as the actual error only happens the second
-# time the objects are built because 'depcomp' has silently messed up the
-# .Po files the first time.
-$MAKE clean
-
-$MAKE >out 2>&1 || { cat out; Exit 1; }
-cat out
-grep 'src/[._]deps' out && Exit 1
-
-:
# Test to make sure that depcomp is not used when it's not installed
# From Pavel Roskin.
-required=gcc
+required=cc
. ./defs || Exit 1
cat >> configure.in << 'END'
: > subdir/foo.c
-# Ignore user CFLAGS.
-unset CFLAGS || :
-
$ACLOCAL
$AUTOMAKE --add-missing
$AUTOCONF
./configure 2>stderr || { cat stderr >&2; Exit 1; }
+cat stderr >&2
test ! -s stderr
:
$AUTOCONF
$AUTOMAKE -a
-# Sanity check: make sure the cache variable we force is used by configure
+# Sanity check: make sure the cache variable we force is used by configure.
grep am_cv_CC_dependencies_compiler_type configure
./configure am_cv_CC_dependencies_compiler_type=dashmstdout
$AUTOCONF
$AUTOMAKE -a
-# Sanity check: make sure the cache variable we force is used by configure
+# Sanity check: make sure the cache variable we force is used by configure.
grep am_cv_CC_dependencies_compiler_type configure
./configure am_cv_CC_dependencies_compiler_type=makedepend
# Check dependency generation (non libtool case).
+required=cc
. ./defs || Exit 1
cat >> configure.in << 'END'
./configure --enable-dependency-tracking
$MAKE
-# Check that dependency tracking works.
if grep 'depmode=none' Makefile; then
- Exit 77
+ skip_ "automatic dependency tracking couldn't be activated"
fi
cd sub2
# Check dependency generation (libtool case).
-required='libtoolize'
+required='cc libtoolize'
. ./defs || Exit 1
cat >> configure.in << 'END'
# Keep this in sync with sister test `depcomp8b.test', which checks the
# same thing for libtool objects.
+required=cc
. ./defs || Exit 1
cat >> configure.in << 'END'
$ACLOCAL
$AUTOMAKE -a
-grep include Makefile.in # for debugging
+grep include Makefile.in # For debugging.
grep 'include.*\./\$(DEPDIR)/foo\.P' Makefile.in
grep 'include.*\./\$(DEPDIR)/bar\.P' Makefile.in
grep 'include.*/\./\$(DEPDIR)' Makefile.in && Exit 1
# Don't reject slower dependency extractors, for better coverage.
./configure --enable-dependency-tracking
$MAKE
-./zardoz
+cross_compiling || ./zardoz
DISTCHECK_CONFIGURE_FLAGS='--enable-dependency-tracking' $MAKE distcheck
# Try again with subdir-objects option.
$ACLOCAL
$AUTOMAKE -a
-grep include Makefile.in # for debugging
+grep include Makefile.in # For debugging.
grep 'include.*\./\$(DEPDIR)/foo\.P' Makefile.in
grep 'include.*[^a-zA-Z0-9_/]sub/\$(DEPDIR)/bar\.P' Makefile.in
$EGREP 'include.*/(\.|sub)/\$\(DEPDIR\)' Makefile.in && Exit 1
# Don't reject slower dependency extractors, for better coverage.
./configure --enable-dependency-tracking
$MAKE
-./zardoz
+cross_compiling || ./zardoz
DISTCHECK_CONFIGURE_FLAGS='--enable-dependency-tracking' $MAKE distcheck
:
# Keep this in sync with sister test `depcomp8a.test', which checks the
# same thing for non-libtool objects.
-required=libtoolize
+required='cc libtoolize'
. ./defs || Exit 1
cat >> configure.in << 'END'
$ACLOCAL
$AUTOMAKE -a
-grep include Makefile.in # for debugging
+grep include Makefile.in # For debugging.
grep 'include.*\./\$(DEPDIR)/foo\.P' Makefile.in
grep 'include.*\./\$(DEPDIR)/bar\.P' Makefile.in
grep 'include.*/\./\$(DEPDIR)' Makefile.in && Exit 1
$ACLOCAL
$AUTOMAKE -a
-grep include Makefile.in # for debugging
+grep include Makefile.in # For debugging.
grep 'include.*\./\$(DEPDIR)/foo\.P' Makefile.in
grep 'include.*[^a-zA-Z0-9_/]sub/\$(DEPDIR)/bar\.P' Makefile.in
$EGREP 'include.*/(\.|sub)/\$\(DEPDIR\)' Makefile.in && Exit 1
+++ /dev/null
-#! /bin/sh
-# Copyright (C) 2011 Free Software Foundation, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2, or (at your option)
-# any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program. If not, see <http://www.gnu.org/licenses/>.
-
-# makedepend should work in VPATH mode.
-
-# Here's the bug: makedepend will prefix VPATH to the object file name,
-# thus the second make will invoke depcomp with object='../../src/foo.o',
-# causing errors such as:
-# touch: cannot touch `../../src/.deps/foo.TPo': No such file or directory
-# makedepend: error: cannot open "../../src/.deps/foo.TPo"
-# ../../depcomp: line 560: ../../src/.deps/foo.TPo: No such file or directory
-
-# We include subfoo only to be sure that we don't remove too much
-# from the object file name.
-
-required='makedepend'
-. ./defs || Exit 1
-
-mkdir src src/sub build
-
-cat >> configure.in << 'END'
-AC_PROG_CC
-AM_PROG_CC_C_O
-AC_CONFIG_FILES([src/Makefile])
-AC_OUTPUT
-END
-
-cat > Makefile.am << 'END'
-SUBDIRS = src
-END
-
-cat > src/Makefile.am << 'END'
-AUTOMAKE_OPTIONS = subdir-objects
-bin_PROGRAMS = foo
-foo_SOURCES = foo.c foo.h sub/subfoo.c
-END
-
-cat > src/foo.h <<EOF
-extern int subfoo (void);
-EOF
-
-cat >src/foo.c <<EOF
-#include "foo.h"
-int main (void)
-{
- return subfoo ();
-}
-EOF
-
-cat >src/sub/subfoo.c <<EOF
-#include "foo.h"
-int subfoo (void)
-{
- return 0;
-}
-EOF
-
-$ACLOCAL
-$AUTOCONF
-$AUTOMAKE -a
-
-cd build
-../configure am_cv_CC_dependencies_compiler_type=makedepend
-
-# Do not error out with the first make, as the forced 'makedepend'
-# depmode might not actually work, but we have overridden the
-# _AM_DEPENDENCIES tests.
-$MAKE || Exit 77
-
-# We must clean and rebuild, as the actual error only happens the second
-# time the objects are built because 'makedepend' has silently messed up
-# the .Po files the first time.
-$MAKE clean
-
-$MAKE >out 2>&1 || { cat out; Exit 1; }
-cat out
-grep 'src/[._]deps' out && Exit 1
-
-:
# Make sure depcomp is found for the distribution.
# From Eric Magnien.
+required=cc
. ./defs || Exit 1
rm -f configure.in
# Test to make sure dependency tracking works with
# --disable-dependency-tracking.
-# Report from Robert Boehne
+# Report from Robert Boehne.
-required=gcc
+required=cc
. ./defs || Exit 1
cat >> configure.in << 'END'
cat > hello.c << 'END'
#include "hello.h"
-int
-main (int argc, char *argv[])
+int main (int argc, char *argv[])
{
printf ("yeah, yeah\n");
return 0;
$AUTOCONF
$AUTOMAKE -a
-# Ignore user CFLAGS.
-CFLAGS=
-export CFLAGS
-
./configure --disable-dependency-tracking
-
$MAKE
+
+:
$ACLOCAL
headers=
-for header in one.h two.h three.h four.h five.h six.h
-do
- headers="$headers $header"
- cat > Makefile.am << END
-bin_PROGRAMS = fred
-fred_SOURCES = fred1.c $headers
+for header in one.h two.h three.h four.h five.h six.h; do
+ headers="$headers $header"
+ unindent > Makefile.am << END
+ bin_PROGRAMS = fred
+ fred_SOURCES = fred1.c $headers
END
-
- $AUTOMAKE
-
- test 1 = `grep '^@AMDEP_TRUE@@am__include@' Makefile.in | wc -l`
+ $AUTOMAKE
+ test 1 = `grep '^@AMDEP_TRUE@@am__include@' Makefile.in | wc -l`
done
+
+:
# Check that _AM_OUTPUT_DEPENDENCY_COMMANDS works with eval-style
# quoting in $CONFIG_FILES, done by newer Autoconf.
+required=cc
. ./defs || Exit 1
cat >>configure.in << END
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check that dependency tracking in various flavours works with VPATH
+# builds. Examples of reported failures follows ...
+
+# <http://lists.gnu.org/archive/html/automake-patches/2011-04/msg00028.html>
+#
+# Here's the bug: makedepend will prefix VPATH to the object file name,
+# thus the second make will invoke depcomp with object='../../src/foo.o',
+# causing errors such as:
+# touch: cannot touch `../../src/.deps/foo.TPo': No such file or directory
+# makedepend: error: cannot open "../../src/.deps/foo.TPo"
+# ../../depcomp: line 560: ../../src/.deps/foo.TPo: No such file or directory
+
+# <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=8473>
+# <http://lists.gnu.org/archive/html/automake-patches/2011-04/msg00079.html>
+#
+# Here's the bug: hp depmode will prefix VPATH to the object file name,
+# thus the second gmake will invoke depcomp with object='../../src/foo.o',
+# causing errors such as (broken on multiple lines for clarity):
+# cpp: "", line 0: error 4066: Cannot create
+# "../../gllib/.deps/nonblocking.TPo" file for
+# "-M../../gllib/.deps/nonblocking.TPo" option.
+# (No such file or directory[errno=2])
+#
+
+# <http://lists.gnu.org/archive/html/automake-patches/2011-04/msg00140.html>
+# <http://lists.gnu.org/archive/html/automake-patches/2011-05/msg00019.html>
+#
+# A partial failure of an earlier version of this test; some bad
+# post-processing of the `*.Po' files led to the following broken
+# contents of `src/sub/.deps/subfoo.Po':
+# > sub/subfoo.o: ../../depmod-data.dir/src/sub/subfoo.c \
+# > ../../depmod-data.dir/src/foo.h
+# > ../../depmod-data.dir/src/sub/subfoo.c \:
+# > ../../depmod-data.dir/src/foo.h:
+# which caused make to die with an error like:
+# "sub/.deps/subfoo.Po:3: *** missing separator. Stop."
+
+required=cc
+. ./defs || Exit 99
+
+# Per each valid depmode three kinds of VPATH, and per each of them
+# a test on "./configure" and one on "make & remake"; so there are
+# 17 * 3 * 2 = 102.
+plan_ 102
+
+ocwd=`pwd` || fatal_ "cannot get current working directory"
+
+# Keep this in sync with the contents of depend.m4.
+all_depmodes=`\
+ sed -n 's/^#*\([a-zA-Z0-9]*\))$/\1/p' < "$top_testsrcdir/lib/depcomp" \
+ | grep -v '^none$'` \
+ && : Turn newlines and tabs into spaces, and strip extra whitespace. \
+ && all_depmodes=`echo $all_depmodes` \
+ && test -n "$all_depmodes" \
+ || fatal_ "can't extract list of valid depmodes"
+
+srctree=depmod-1.0
+mkdir $srctree
+mv missing install-sh depcomp configure.in $srctree
+cd $srctree
+
+mkdir src src/sub
+
+cat >> configure.in << 'END'
+AC_CONFIG_FILES([src/Makefile])
+AC_PROG_CC
+AM_PROG_CC_C_O
+AC_OUTPUT
+END
+
+echo 'SUBDIRS = src' > Makefile.am
+
+cat > src/Makefile.am << 'END'
+AUTOMAKE_OPTIONS = subdir-objects
+bin_PROGRAMS = foo
+foo_SOURCES = foo.c foo.h sub/subfoo.c
+END
+
+echo 'extern int subfoo (void);' > src/foo.h
+
+cat > src/foo.c << 'END'
+#include "foo.h"
+int main (void)
+{
+ return subfoo ();
+}
+END
+
+# We include subfoo only to be sure that we don't remove too much
+# from the object file name.
+cat > src/sub/subfoo.c << 'END'
+#include "foo.h"
+int subfoo (void)
+{
+ return 0;
+}
+END
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+# Sanity check: make sure the cache variable we force is used
+# by configure.
+grep am_cv_CC_dependencies_compiler_type configure \
+ || fatal_ "configure lacks required cache variable"
+
+cd "$ocwd" || fatal_ "cannot chdir back to top directory"
+
+longpath=this-is/a-path/which-have/quite-a/long_long_name
+$srctree/install-sh -d $longpath || fatal_ "cannot create long path"
+cp -pR $srctree/* $longpath || fatal_ "cannot populate long path"
+cp -pR $srctree/* . || fatal_ "cannot populate top-level directory"
+
+for vpath in simple long absolute; do
+ for depmode in $all_depmodes; do
+ case $vpath in
+ simple)
+ srcdir=..;;
+ long)
+ srcdir=../$longpath;;
+ absolute)
+ srcdir=$ocwd;;
+ *)
+ fatal_ "bad \$vpath value '$vpath'";;
+ esac
+
+ pfx="$depmode [$vpath VPATH]"
+
+ mkdir $depmode-$vpath.d
+ cd $depmode-$vpath.d
+
+ if "$srcdir"/configure am_cv_CC_dependencies_compiler_type=$depmode
+ then
+ r0='ok'
+ else
+ r0='not ok'
+ fi
+ result_ "$r0" "$pfx configure" \
+
+ # Do not fail with the first call to make fails, as the depmode
+ # we've forced might not actually work, but we have overridden the
+ # _AM_DEPENDENCIES tests.
+ r1='not ok'
+ if $MAKE; then
+ # We must clean and rebuild, as the actual error only happens the
+ # second time the objects are built because 'depcomp' has silently
+ # messed up the .Po files the first time.
+ $MAKE clean \
+ && { e=0; $MAKE >output 2>&1 || e=1; cat output; test $e -eq 0; } \
+ && { if grep 'src/[._]deps' output; then false; else :; fi; } \
+ && r1='ok'
+ result_ "$r1" "$pfx make & remake"
+ else
+ skip_ -r "can't force depmode" "$pfx make & remake"
+ fi
+
+ cd "$ocwd" || fatal_ "cannot chdir back to top directory"
+
+ # Remove subdirectories for tests that have passed, to avoid ending up
+ # with a too big test directory. This is especially important since,
+ # on each and every system, at least some tests in this script are
+ # expected to be skipped (which ones exactly depends on the system),
+ # thus causing the test directory not to be removed when the script
+ # terminates.
+ case $keep_testdirs,$r0,$r1 in
+ ,ok,ok) rm_rf_ $depmode-$vpath.d;;
+ *) : For lesser shells with broken 'set -e';;
+ esac
+
+ done
+done
+
+:
$am_init_automake
AC_CONFIG_FILES([Makefile])
END
- cat configure.in # might be useful for debugging
+ cat configure.in # For debugging.
$ACLOCAL 2>stderr && { cat stderr >&2; Exit 1; }
cat stderr >&2
grep "^configure\\.in:.* $errmsg" stderr
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Check dirlist support.
-# This test relies on m4/dirlist
+# This test relies on m4/dirlist.
. ./defs || Exit 1
$ACLOCAL
$AUTOCONF
-# there should be no m4_include in aclocal.m4, even tho m4/dirlist contains
-# `./dirlist-test' as a relative directory. Only -I directories are subject
-# to file inclusion.
+# There should be no m4_include in aclocal.m4, even though m4/dirlist
+# contains `./dirlist-test' as a relative directory. Only -I directories
+# are subject to file inclusion.
grep m4_include aclocal.m4 && Exit 1
grep 'GUILE-VERSION' configure
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Check dirlist globbing support.
-# This test relies on m4/dirlist
+# This test relies on m4/dirlist.
. ./defs || Exit 1
$ACLOCAL
$AUTOCONF
-# there should be no m4_include in aclocal.m4, even tho m4/dirlist contains
-# `./dirlist-test' as a relative directory. Only -I directories are subject
-# to file inclusion.
+# There should be no m4_include in aclocal.m4, even though m4/dirlist
+# contains `./dirlist-test' as a relative directory. Only -I directories
+# are subject to file inclusion.
grep m4_include aclocal.m4 && Exit 1
grep 'GUILE-VERSION' configure
. ./defs || Exit 1
-set -e
-
cat >> configure.in << 'END'
AC_ARG_ENABLE([success], [], [success=$enableval], [success=no])
AS_IF([test $success = yes && test "$sentence" = 'it works :-)'],
. ./defs || Exit 1
-set -e
-
cat >> configure.in << 'END'
AC_CONFIG_SUBDIRS([subpkg])
if test $dc != ok || test $am_dc != ok; then
. ./defs || Exit 1
-set -e
-
cat >> configure.in << 'END'
AC_ARG_ENABLE([success], [], [success=$enableval], [success=no])
AS_IF([test $success = yes && test "$sentence" = 'it works :-)'],
. ./defs || Exit 1
-set -e
-
cat >> configure.in << 'END'
AC_OUTPUT
END
. ./defs || Exit 1
-set -e
-
cat >> configure.in << 'END'
AC_CONFIG_SUBDIRS([subpkg])
AC_OUTPUT
# Make sure the suggested `distcleancheck_listfiles' in the manual works.
# The example Makefile.am we use is from the FAQ entry `distcleancheck'.
+required=cc
. ./defs || Exit 1
cat >>configure.in << 'END'
n
}' -e 's/$/ /' -e '/^DIST_COMMON =/p' Makefile.in > dc.txt
-cat dc.txt # might be useful for debugging
+cat dc.txt # For debugging.
$FGREP ' compile ' dc.txt
$FGREP ' depcomp ' dc.txt
echo 'I bet you are reading me.' > README
END
-# Files required by --gnu
+# Files required by `--gnu'.
: > NEWS
: > AUTHORS
: > ChangeLog
n
}' -e '/^DIST_COMMON =/ p' Makefile.in > dc.txt
-cat dc.txt # might be useful for debugging
+cat dc.txt # For debugging.
test 1 = `grep tests dc.txt | wc -l`
grep configure dc.txt
extract_distcommon Makefile.in > top.txt
extract_distcommon tests/Makefile.in > inner.txt
-# might be useful for debugging
+# Might be useful for debugging.
cat top.txt
cat inner.txt
n
}' -e 's/$/ /' -e '/^DIST_COMMON =/p' Makefile.in > dc.txt
-cat dc.txt # might be useful for debugging
+cat dc.txt # For debugging.
$FGREP ' compile ' dc.txt
$FGREP ' depcomp ' dc.txt
echo text > file
-ln -s file lnk || {
- echo "$me: cannot create symlinks to files" >&2
- Exit 77
-}
+ln -s file lnk || skip_ "cannot create symlinks to files"
mkdir A
mkdir B
AC_OUTPUT
END
-echo "me = $me" > Makefile.am # for better failure messages
+echo "me = $me" > Makefile.am # For better failure messages.
cat >> Makefile.am << 'END'
EXTRA_DIST = lnk B/bbb bar1 bar2 bar3 quux
.PHONY: test
lnka=${lnk_base}__aaa
lnkb=${lnk_base}__bbb
-ln -s nonesuch $lnk1 || {
- echo "$me: cannot create broken symlinks" >&2
- Exit 77
-}
+ln -s nonesuch $lnk1 || skip_ "cannot create broken symlinks"
ln -s "`pwd`/nonesuch" $lnk2
$ACLOCAL
$AUTOCONF
-ls -l # for debugging
+ls -l # For debugging.
# Don't try to use "make -k", because some botched make implementations
# (HP-UX, IRIX) might still exit on the first error in this situations.
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Test of names in tar file.
-# From Rainer Orth
+# From Rainer Orth.
+required=cc
. ./defs || Exit 1
cat >> configure.in << 'END'
$AUTOCONF
./configure
-cat config.h # debug info
+cat config.h # For debugging.
grep '^ */\* *# *undef *WITH_DMALLOC *\*/ *$' config.h
./configure --without-dmalloc
-cat config.h # debug info
+cat config.h # For debugging.
grep '^ */\* *# *undef *WITH_DMALLOC *\*/ *$' config.h
./configure --with-dmalloc
-cat config.h # debug info
+cat config.h # For debugging.
grep '^# *define *WITH_DMALLOC *1 *$' config.h
$MAKE check-with-dmalloc
echo 'AM_CONDITIONAL(FOO, true)' >> configure.in
cat > Makefile.am << 'END'
-# flag to tell us if apache dir is a source distribution
+# Flag to tell us if apache dir is a source distribution.
APACHE_DIR_IS_SRC = @APACHE_DIR_IS_SRC@
-# we only need to descend into the c dir if we're doing a 1.3 DSO configuration
+# We only need to descend into the c dir if we're doing a 1.3 DSO
+# configuration.
ifeq ("${APACHE_DIR_IS_SRC}","yes")
SUBDIRS = java
else
$ACLOCAL
AUTOMAKE_fails
-grep 'Makefile.am:7:.*else without if' stderr
+grep '^Makefile\.am:8:.*else without if' stderr
# Also make sure the old definitions of bin_PROGRAMS and friend
# aren't left around. Report from Jim Meyering.
+required=cc
. ./defs || Exit 1
cat >> configure.in << 'END'
# Make sure $(EXEEXT) is appended to programs and to tests that are
# programs, but not to @substitutions@.
+required=cc
. ./defs || Exit 1
cat >> configure.in << 'END'
$ACLOCAL
$AUTOCONF
-$AUTOMAKE
+$AUTOMAKE --add-missing --copy
./configure
$MAKE print-bin > output
cat output
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2003, 2006, 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Test that EXTRA_PROGRAMS doesn't get removed because it is empty.
+# This check hs been introduced in commit `Release-1-9-254-g9d0eaef'
+# into the former test `subst2.test'.
+
+. ./defs || Exit 1
+
+cat >> configure.in << 'END'
+AC_CONFIG_FILES([Makefile2 Makefile3])
+AC_SUBST([prog])
+AC_PROG_CC
+AC_OUTPUT
+END
+
+cat > Makefile.am <<'END'
+EXTRA_PROGRAMS =
+END
+
+cat > Makefile2.am <<'END'
+bin_PROGRAMS = a @prog@ b
+EXTRA_PROGRAMS =
+END
+
+cat > Makefile3.am <<'END'
+empty =
+EXTRA_PROGRAMS = $(empty)
+END
+
+$ACLOCAL
+$AUTOMAKE
+
+grep '^EXTRA_PROGRAMS = *$' Makefile.in
+grep '^EXTRA_PROGRAMS = *$' Makefile2.in
+# Be laxer here, since EXTRA_PROGRAMS might be internally rewritten
+# by Automake when it contains references to other variables.
+grep '^EXTRA_PROGRAMS =' Makefile3.in
+
+:
--- /dev/null
+#! /usr/bin/env perl
+# Extract the testsuite summary generated by the parallel-tests harness
+# from the output of "make check".
+
+use warnings FATAL => 'all';
+use strict;
+
+my $br = '=' x 76;
+my @sections = ('');
+while (<>)
+ {
+ push @sections, $_, '' if /$br/;
+ $sections[-1] .= $_ if !/$br/;
+ }
+print @sections[1..$#sections-1];
# Test EXTRA_*_DEPENDENCIES. See extradep2 for libtool variant.
+required=cc
. ./defs || Exit 1
cat >> configure.in << 'END'
./configure cond=yes
-# hypotheses:
-# EXTRA_*_DEPENDENCIES are honored.
-# conditionals and substitutions are honored.
-# *_DEPENDENCIES are not overwritten by their EXTRA_* counterparts.
+# Hypotheses:
+# - EXTRA_*_DEPENDENCIES are honored.
+# - Conditionals and substitutions are honored.
+# - *_DEPENDENCIES are not overwritten by their EXTRA_* counterparts.
: >foodep
: >foodep2
# Test EXTRA_*_DEPENDENCIES, libtool version; see extradep.test.
-required=libtoolize
+required='cc libtoolize'
. ./defs || Exit 1
cat >> configure.in << 'END'
./configure
-# hypothesis: EXTRA_*_DEPENDENCIES are honored.
+# Hypothesis: EXTRA_*_DEPENDENCIES are honored.
: >foodep
: >foodep2
EXTRA_DIST = 12345678
END
-(for i in 1 2 3 4 5 6 7 8 9
-do
- mkdir -p 12345678 || Exit 77
- cd 12345678
- touch x
-done) || Exit 77
+(for i in 1 2 3 4 5 6 7 8 9; do
+ mkdir -p 12345678 && cd 12345678 && touch x || Exit 1
+done) || skip_ "failed to create deep directory hierarchy"
# AIX 5.3 `cp -R' is too buggy for `make dist'.
-cp -R 12345678 t || Exit 77
+cp -R 12345678 t \
+ || skip_ "'cp -R' failed to copy deep directory hierarchy"
$ACLOCAL
$AUTOCONF
./configure
$MAKE distcheck
-(for i in 1 2 3 4 5 6 7 8 9 10 11
-do
- mkdir -p 12345678 || Exit 77
- cd 12345678
- touch x
-done) || Exit 77
+(for i in 1 2 3 4 5 6 7 8 9 10 11; do
+ mkdir -p 12345678 && cd 12345678 && touch x || Exit 1
+done) || skip_ "failed to create deeper directory hierarchy"
$MAKE dist 2>stderr && { cat stderr >&2; Exit 1; }
cat stderr >&2
grep 'filenames are too long' stderr
test 2 = `grep 12345678 stderr | wc -l`
+
:
subdirname='cnfsubdir'
cat >>configure.in <<END
-AC_CONFIG_SUBDIRS([${subdirname}])
+AC_CONFIG_SUBDIRS([$subdirname])
AC_OUTPUT
END
cat >Makefile.am <<END
AUTOMAKE_OPTIONS = filename-length-max=99
-SUBDIRS = ${subdirname}
+SUBDIRS = $subdirname
END
-mkdir ${subdirname} || Exit 1
+mkdir $subdirname
-cat >> ${subdirname}/configure.in <<EOF
-AC_INIT([${subdirname}], [1.0])
+cat >> $subdirname/configure.in <<EOF
+AC_INIT([$subdirname], [1.0])
AM_INIT_AUTOMAKE
AC_CONFIG_FILES([Makefile])
AC_OUTPUT
EOF
-cat >${subdirname}/Makefile.am <<'END'
+cat >$subdirname/Makefile.am <<'END'
AUTOMAKE_OPTIONS = filename-length-max=99
EXTRA_DIST = 12345678
END
-(cd ${subdirname}; for i in 1 2 3 4 5 6 7 8
-do
- mkdir -p 12345678 || Exit 77
- cd 12345678
- touch x
-done)
+(cd $subdirname || Exit 1
+for i in 1 2 3 4 5 6 7 8; do
+ mkdir -p 12345678 && cd 12345678 && touch x || Exit 1
+done) || skip_ "failed to create deep directory hierarchy"
# AIX 5.3 `cp -R' is too buggy for `make dist'.
-cp -R ${subdirname} t || Exit 77
-
-for init_dir in ${subdirname} .; do
- (
- cd ${init_dir} || Exit 1
- $ACLOCAL
- $AUTOCONF
- $AUTOMAKE
- ) || Exit 1
-done
+cp -R $subdirname t \
+ || skip_ "'cp -R' failed to copy deep directory hierarchy"
+
+cd $subdirname
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE
+cd ..
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE
+
./configure
$MAKE distcheck
-(cd ${subdirname}; for i in 1 2 3 4 5 6 7 8 9
-do
- mkdir -p 12345678 || Exit 77
- cd 12345678
- touch x
-done)
+(cd $subdirname || Exit 1
+for i in 1 2 3 4 5 6 7 8 9; do
+ mkdir -p 12345678 && cd 12345678 && touch x || Exit 1
+done) || skip_ "failed to create deeper directory hierarchy"
$MAKE dist 2>stderr && { cat stderr >&2; Exit 1; }
cat stderr >&2
grep 'filenames are too long' stderr
test 1 = `grep 12345678 stderr | wc -l`
+
:
echo zot > foo
cp foo foo2
-ln -s foo2 install-sh || {
- echo "$me: cannot create symlinks to files" >&2
- Exit 77
-}
+ln -s foo2 install-sh || skip_ "cannot create symlinks to files"
: > Makefile.am
grep '.\$(FCLINK)' Makefile.in && Exit 1
$AUTOCONF
-# configure may Exit 77 if no compiler is found,
+# ./configure may exit with status 77 if no compiler is found,
# or if the compiler cannot compile Fortran 90 files).
./configure
$MAKE
$AUTOCONF
# This test requires Libtool >= 2.0. Earlier Libtool does not
-# have the LT_PREREQ macro to cause autoconf failure, so let's
-# skip in this case:
-grep LT_PREREQ configure && Exit 77
+# have the LT_PREREQ macro to cause autoconf failure.
+grep LT_PREREQ configure && skip_ "libtool is too old (probably < 2.0)"
# Ensure we use --tag for f90, too.
grep " --tag=FC" Makefile.in
-# configure may Exit 77 if no compiler is found,
+# ./configure may exit with status 77 if no compiler is found,
# or if the compiler cannot compile Fortran 90 files).
./configure
$MAKE
+++ /dev/null
-#! /bin/sh
-# Generate parallel-tests.am.
-#
-# Copyright (C) 2009, 2010 Free Software Foundation, Inc.
-
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2, or (at your option)
-# any later version.
-
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-
-# You should have received a copy of the GNU General Public License
-# along with this program. If not, see <http://www.gnu.org/licenses/>.
-
-# For each test in the TESTS list in this Makefile.am file, that itself
-# tests features of the TESTS automake interface, generate a sibling
-# test that does likewise, but with the option `parallel-tests' enabled.
-# Individual tests can prevent the creation of such a sibling by
-# explicitly setting the `$parallel_tests' variable to either "yes" or
-# "no". The rationale for this is that if the variable is set to "yes",
-# the test already uses the `parallel-tests' option, so that a sibling
-# would be just a duplicate; while if the variable is set to "no", the
-# test doesn't support, or is not meant to run with, the `parallel-tests'
-# option, and forcing it to do so in the sibling would likely cause a
-# spurious failure.
-
-set -e
-
-tests=`sed -n '/^TESTS =/,/^$/s/\(.*\.test\).*/\1/p' Makefile.am`
-{
- grep -l '^TESTS ' $tests
- grep -l ' TESTS ' $tests
-} |
-grep -v '.-p\.test' |
-LC_ALL=C sort -u |
-while read tst; do
- if grep '^[^#]*parallel-tests' $tst >/dev/null \
- || grep "parallel_tests=" $tst >/dev/null
- then :; else echo $tst; fi;
-done |
-sed -e 's/\.test$//' |
-{
- echo "## Generated by gen-parallel-tests. Edit Makefile.am instead of this."
- echo "parallel_tests ="
- # Since `foo-p.test' sources `foo.test', `foo-p.log' also depends on
- # `foo.test'.
- sed -e '
- h
- s/^/parallel_tests += /
- s/$/-p.test/
- p
- x
- s/.*/&-p.log: &.test/
- '
-}
--- /dev/null
+#! /bin/sh
+# Generate wrap-tests.am.
+#
+# Copyright (C) 2009, 2010, 2011 Free Software Foundation, Inc.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+set -e
+set -u
+
+MAKE=${MAKE-make}
+PERL=${PERL-perl}
+
+tests=`$MAKE -s -f list-of-tests.mk print-list-of-tests` \
+ && tests=`echo $tests` && test -n "$tests" \
+ || { echo "$0: cannot get list of tests" >&2; exit 1; }
+
+st=0
+for tst in $tests; do
+ test -f $tst || { echo "$0: test \`$tst' not found" >&2; st=1; }
+done
+test $st -eq 0 || exit 1
+
+add_wrapper_type () { wrapper_type_list="$wrapper_type_list $*"; }
+wrapper_type_list="" # Avoid interferences from the environment.
+
+#
+# For each test script in the Automake testsuite that itself tests features
+# of the TESTS automake interface, define a sibling test that does likewise,
+# but with the option `parallel-tests' enabled.
+#
+# A test is considered a candidate for sibling-generation if any
+# Makefile.am generated by it define the TESTS variable.
+#
+# Individual tests can prevent the creation of such a sibling by explicitly
+# setting the `$parallel_tests' variable to either "yes" or "no". The
+# rationale for this is that if the variable is set to "yes", the test
+# already uses the `parallel-tests' option, so that a sibling would be just
+# a duplicate; while if the variable is set to "no", the test doesn't
+# support, or is not meant to run with, the `parallel-tests' option, and
+# forcing it to do so in the sibling would likely cause a spurious failure.
+#
+get_list_for_pt ()
+{
+ { grep -l '^TESTS *=' $*; grep -l ' TESTS *=' $*; } \
+ | LC_ALL=C sort -u \
+ | while read tst; do
+ grep '^[^#]*parallel-tests' $tst >/dev/null && continue
+ grep "parallel_tests=" $tst >/dev/null && continue
+ echo $tst
+ done
+}
+defines_for_pt='--define parallel_tests "yes"'
+add_wrapper_type "pt"
+
+#
+# For each test script in the Automake testsuite that tests features of
+# one or more automake-provided shell script from the `lib/' subdirectory
+# by running those scripts directly (i.e., not thought make calls and
+# automake-generated makefiles), define a sibling test that does likewise,
+# but running the said script with the configure-time $SHELL instead of
+# the default system shell /bin/sh.
+#
+# A test is considered a candidate for sibling-generation if it calls the
+# `get_shell_script' function anywhere.
+#
+# Individual tests can prevent the creation of such a sibling by explicitly
+# setting the `$test_prefer_config_shell' variable to either "yes" or "no".
+# The rationale for this is that if the variable is set to "yes", the test
+# already uses $SHELL, so that a sibling would be just a duplicate; while
+# if the variable is set to "no", the test doesn't support, or is not meant
+# to use, $SHELL to run the script under testing, and forcing it to do so
+# in the sibling would likely cause a spurious failure.
+#
+get_list_for_cs ()
+{
+ { grep -l '^get_shell_script ' $*; grep -l ' get_shell_script ' $*; } \
+ | LC_ALL=C sort -u \
+ | while read tst; do
+ grep "test_prefer_config_shell=" $tst >/dev/null || echo $tst
+ done
+}
+defines_for_cs='--define test_prefer_config_shell "yes"'
+add_wrapper_type "cs"
+
+# Tests on tap support should be run with both the perl and awk
+# implementations of the TAP driver.
+get_list_for_pltap ()
+{
+ {
+ grep -l '[ /]tap-setup\.sh' $*;
+ grep -l '^fetch_tap_driver$' $*;
+ grep -l '^fetch_tap_driver ' $*;
+ grep -l ' fetch_tap_driver$' $*;
+ grep -l ' fetch_tap_driver ' $*;
+ } | LC_ALL=C sort -u \
+ | while read tst; do
+ grep "am_tap_implementation=" $tst >/dev/null || echo $tst
+ done
+}
+defines_for_pltap='--define am_tap_implementation "perl"'
+add_wrapper_type "pltap"
+
+echo '## Generated by gen-wrapped-tests. DO NOT EDIT.'
+echo 'wrap_TESTS ='
+echo 'wrap_LOGS ='
+echo
+
+# NOTE: the following is not really right, since cannot compose wrapping
+# of tests matching more than one condition. Still, there should be no
+# such test at the moment, so the limitation is acceptable. Also, if
+# this invariant stops holding, the check `check-no-repeated-test-name'
+# in Makefile.am will start failing.
+
+for wtype in $wrapper_type_list; do
+ eval defines=\${defines_for_$wtype}
+ get_list_for_$wtype $tests \
+ | wtype="$wtype" defines="$defines" $PERL -w -e '
+ use strict;
+ use warnings FATAL => "all";
+
+ my $wtype = $ENV{"wtype"};
+
+ # Suffix for a generated wrapped test, from the suffix of a
+ # hand-written test.
+ sub gen_suffix ($) { return $_[0] . "_" . $wtype; }
+
+ HANDLE_SUFFIX :
+ {
+ my %have_suffix;
+ sub handle_if_new_suffix ($)
+ {
+ my $suf = shift;
+ return
+ if $have_suffix{$suf};
+ $have_suffix{$suf} = 1;
+ print "\n";
+ print "TEST_EXTENSIONS += ." . gen_suffix ($suf) . "\n";
+ my $uc_orig_suf = uc ($suf);
+ my $uc_wrap_suf = uc ($suf) . "_" . uc ($wtype);
+ print <<EOF;
+
+${uc_wrap_suf}_LOG_DRIVER = \\
+ \$(${uc_orig_suf}_LOG_DRIVER)
+
+AM_${uc_wrap_suf}_LOG_DRIVER_FLAGS = \\
+ \$(AM_${uc_orig_suf}_LOG_DRIVER_FLAGS) \\
+ \$(${uc_orig_suf}_LOG_DRIVER_FLAGS)
+
+${uc_wrap_suf}_LOG_COMPILER = \\
+ \$(LOG_COMPILER) \$(srcdir)/wrap-tests.sh \\
+ --type $wtype $ENV{"defines"}
+
+EOF
+ }
+ }
+ while (<>)
+ {
+ chomp;
+ if (/^(.*)\.(test|tap)$/)
+ {
+ my ($bas, $suf) = ($1, $2);
+ handle_if_new_suffix ($suf);
+ print "wrap_TESTS += $bas-w." . gen_suffix ($suf) . "\n";
+ print "wrap_LOGS += $bas-w.log\n";
+ print "$bas-w.log: $bas.$suf\n";
+ }
+ else
+ {
+ die "invalid test name `$_`";
+ }
+ }
+ '
+done
grep ':.*nonexistent' stderr
# ... but this should not.
-AUTOMAKE_run 0 -Wnonexistent --help
+AUTOMAKE_run -Wnonexistent --help
grep ':.*nonexistent' stderr && Exit 1
grep ':.*nonexistent' stderr
# ... but this should not.
-AUTOMAKE_run 0 --nonexistent --help
+AUTOMAKE_run --nonexistent --help
grep ':.*nonexistent' stderr && Exit 1
:
$ACLOCAL
-# po/ and intl/ are required
+# po/ and intl/ are required.
AUTOMAKE_fails --add-missing
grep 'AM_GNU_GETTEXT.*SUBDIRS' stderr
echo 'SUBDIRS = po intl' >Makefile.am
$AUTOMAKE --add-missing
-# Make sure distcheck runs ./configure --with-included-gettext
+# Make sure distcheck runs `./configure --with-included-gettext'.
grep 'with-included-gettext' Makefile.in
# `SUBDIRS = po intl' isn't required if po/ doesn't exist.
echo 'SUBDIRS = po' >Makefile.am
mkdir po
-# if aclocal fails, assume the gettext macros are too old and do not
+# If aclocal fails, assume the gettext macros are too old and do not
# define AM_GNU_GETTEXT_INTL_SUBDIR.
-$ACLOCAL || Exit 77
+$ACLOCAL || skip_ "your gettext macros are probably too old"
# config.rpath is required.
: >config.rpath
# Test to ensure std-options checking is correct.
-required=gcc
+required='cc native'
. ./defs || Exit 1
cat >> configure.in << 'END'
grep-stderr:
grep 'pfubar$(EXEEXT) does not support' stderr
grep 'pfubar3$(EXEEXT) does not support' stderr
- grep 'pscriptnok.sh does not support' stderr
+ grep 'pscriptnok\.sh does not support' stderr
## Only three failures please.
test `grep 'does not support --help' stderr | wc -l` = 3
test `grep 'does not support --version' stderr | wc -l` = 3
test ! -f ../inst-dir/bin/fine$(EXEEXT)
END
-echo 'int main () { return 0; }' > fubar.c
+cat > fubar.c <<'END'
+int main (void)
+{
+ return 0;
+}
+END
cat > fine.c << 'END'
#include <stdio.h>
-int
-main ()
+int main (void)
{
puts ("Which version? Which usage?");
return 0;
# Check that AM_INSTALLCHECK_STD_OPTIONS_EXEMPT works.
-required=gcc
+required=cc
. ./defs || Exit 1
cat >> configure.in << 'END'
grep-stderr:
grep 'sub/pnok$(EXEEXT) does not support' stderr
- grep 'sub/pnok.sh does not support' stderr
+ grep 'sub/pnok\.sh does not support' stderr
## Only two failures please.
test `grep 'does not support --help' stderr | wc -l` = 2
test `grep 'does not support --version' stderr | wc -l` = 2
END
-echo 'int main () { return 0; }' > nok.c
+cat > nok.c <<'END'
+int main (void)
+{
+ return 0;
+}
+END
mkdir sub
AM_MAINTAINER_MODE
END
-cat configure.in
+cat configure.in # For debugging.
$ACLOCAL
$AUTOCONF --force
./configure --help >stdout || { cat stdout; Exit 1; }
$EGREP '^ *--enable-maintainer-mode( |$)' stdout
$FGREP ' enable make rules' stdout
-rm -rf autom4te*.cache # just to be sure
+rm -rf autom4te*.cache # Just to be sure.
sed 's/\(AM_MAINTAINER_MODE\).*/\1([disable])/' configure.in >t
mv -f t configure.in
-cat configure.in
+cat configure.in # For debugging.
$ACLOCAL
$AUTOCONF --force
./configure --help >stdout || { cat stdout; Exit 1; }
$EGREP '^ *--enable-maintainer-mode( |$)' stdout
$FGREP ' enable make rules' stdout
-rm -rf autom4te*.cache # just to be sure
+rm -rf autom4te*.cache # Just to be sure.
sed 's/\(AM_MAINTAINER_MODE\).*/\1([enable])/' configure.in >t
mv -f t configure.in
-cat configure.in
+cat configure.in # For debugging.
$ACLOCAL
$AUTOCONF --force
./configure --help >stdout || { cat stdout; Exit 1; }
END
cat > Makefile.am <<END
-## so that install-sh will be required
+# So that install-sh will be required.
pkgdata_DATA =
END
ACLOCAL=$original_ACLOCAL
AUTOMAKE=$original_AUTOMAKE
-escape_dots () { sed 's/\./\\./g'; } # avoid issues with `\' in backquotes
+escape_dots () { sed 's/\./\\./g'; } # Avoid issues with `\' in backquotes.
apiversion_rx=`echo "$APIVERSION" | escape_dots`
$ACLOCAL --version --help >stdout || { cat stdout; Exit 1; }
+++ /dev/null
-#! /bin/sh
-# Copyright (C) 2004, 2011 Free Software Foundation, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2, or (at your option)
-# any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program. If not, see <http://www.gnu.org/licenses/>.
-
-# Test to make sure automake correctly recognizes presence of
-# AC_CANONICAL_(BUILD|HOST|SYSTEM|TARGET), and creates config.* in all
-# those cases.
-# From Norman Gray.
-
-. ./defs || Exit 1
-
-: >Makefile.am
-
-$ACLOCAL
-$AUTOMAKE --add-missing
-test ! -f config.guess
-test ! -f config.sub
-
-# Test all four of the AC_CANONICAL_* targets, including _SYSTEM, which is
-# supported but deprecated by autoconf.
-for macro in AC_CANONICAL_BUILD AC_CANONICAL_HOST \
- AC_CANONICAL_SYSTEM AC_CANONICAL_TARGET
-do
- rm -rf autom4te.cache config.sub config.guess
-
- cat >configure.in<<EOF
-AC_INIT([hosts], [1.0])
-$macro
-AM_INIT_AUTOMAKE
-AC_CONFIG_FILES([Makefile])
-EOF
-
- $ACLOCAL
- $AUTOMAKE --add-missing
-
- # Show the files which were installed
- ls
-
- test -f config.guess
- test -f config.sub
-done
for ac_init_args in '' '([x])'; do
for am_init_args in '' '([1.10])'; do
rm -rf aclocal.m4 autom4te*.cache
- cat >configure.in <<END
-AC_INIT$ac_init_args
-AM_INIT_AUTOMAKE$am_init_args
+ unindent >configure.in <<END
+ AC_INIT$ac_init_args
+ AM_INIT_AUTOMAKE$am_init_args
END
- cat configure.in # might be useful for debugging
+ cat configure.in # For debugging.
# The error message should mention AC_INIT, not AC_PACKAGE_VERSION.
($ACLOCAL && $AUTOCONF) 2>stderr && { cat stderr >&2; Exit 1; }
cat stderr >&2
. ./defs || Exit 1
+echo AC_OUTPUT >> configure.in
+
cat > Makefile.am << 'END'
pkgdata_DATA =
-magic:
- @echo $(DISTFILES)
+.PHONY: test
+test: distdir
+ find $(distdir) ;: For debugging.
+ echo ' ' $(DISTFILES) ' ' | grep '[ /]install-sh '
+ echo ' ' $(DIST_COMMON) ' ' | grep '[ /]install-sh '
+ test -f $(distdir)/install-sh
END
$ACLOCAL
$AUTOMAKE
+$AUTOCONF
-$FGREP -v @SET_MAKE@ Makefile.in > Makefile.sed
-$MAKE -s -f Makefile.sed SHELL=$SHELL magic | grep install-sh
+./configure
+$MAKE test
:
chmod 000 Makefile.am
# On some systems (like DOS and Windows), files are always readable.
-test ! -r Makefile.am || Exit 77
+test ! -r Makefile.am || skip_ "cannot drop file read permissions"
./configure
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Test for PR 203.
+# == Report ==
+# Some standard targets are missing `-local' hooks. For instance,
+# installdirs is missing this. Ideally this would be an automatic
+# feature of any exported target.
. ./defs || Exit 1
# be installed there.
# This test exercises the libtool code paths.
-required=libtoolize
+required='cc libtoolize'
. ./defs || Exit 1
cat >>configure.in <<'END'
# nothing should be installed there.
# This test exercises the prog and libs code paths.
+required=cc
. ./defs || Exit 1
cat >>configure.in <<'END'
required='makeinfo-html tex texi2dvi'
. ./defs || Exit 1
-dvips --help || Exit 77
-pdfetex --help || pdftex --help || Exit 77
+dvips --help \
+ || skip_ "dvips is missing"
+pdfetex --version || pdftex --version \
+ || skip_ "pdeftex and pdftex are both missing"
cat >>configure.in <<'END'
AC_OUTPUT
for file in info1.info
do
chmod a-r $file
- test ! -r $file || Exit 77
+ test ! -r $file || skip_ "cannot drop file read permissions"
$MAKE install-data && Exit 1
chmod u+r $file
done
END
for n in 1 2 3; do
- cat >java$n.java <<END
-class java$n
-{
-}
-END
+ echo "class java$n { }" >java$n.java
done
$ACLOCAL
for file in java1.class
do
chmod a-r $file
- test ! -r $file || Exit 77
+ test ! -r $file || skip_ "cannot drop file read permissions"
$MAKE install-data && Exit 1
chmod u+r $file
done
# This is the libtool sister test of instfail.test
-required='libtool libtoolize'
+required='cc libtool libtoolize'
. ./defs || Exit 1
cat >>configure.in <<END
for file in liblt1.la libltn1.la
do
chmod a-r $file
- test ! -r $file || Exit 77
+ test ! -r $file || skip_ "cannot drop file read permissions"
$MAKE install-exec && Exit 1
chmod u+r $file
done
# This test has a few sister tests, for java, info, libtool.
+required=cc
. ./defs || Exit 1
cat >>configure.in <<END
for file in lib1.a libn1.a
do
chmod a-r $file
- test ! -r $file || Exit 77
+ test ! -r $file || skip_ "cannot drop file read permissions"
$MAKE install-exec && Exit 1
chmod u+r $file
done
limit=2500
subdir=long_subdir_name_with_many_characters
nfiles=81
-
-# Let's use `seq' if available, it's faster than the loop.
-list=`(seq 1 $nfiles) 2>/dev/null || {
- i=1
- while test $i -le $nfiles; do
- echo $i
- i=\`expr $i + 1\`
- done; }`
+list=`seq_ 1 $nfiles`
sed "s|@limit@|$limit|g" >myinstall.in <<'END'
#! /bin/sh
END
for n in $list; do
- cat >>Makefile.am <<END
-man_MANS += page$n.1
-man3_MANS += page$n.man
-notrans_man_MANS += npage$n.1
-notrans_man3_MANS += npage$n.man
+ unindent >>Makefile.am <<END
+ man_MANS += page$n.1
+ man3_MANS += page$n.man
+ notrans_man_MANS += npage$n.1
+ notrans_man3_MANS += npage$n.man
END
echo >page$n.1
echo >page$n.man
for file in page3.1 page$nfiles.1 npage3.1 npage$nfiles.1; do
chmod a-r $srcdir/$file
- test ! -r $srcdir/$file || Exit 77
+ test ! -r $srcdir/$file || skip_ "cannot drop file read permissions"
$MAKE install-man1 && Exit 1
chmod u+r $srcdir/$file
done
limit=2500
subdir=long_subdir_name_with_many_characters
nfiles=81
-
-list=`(seq 1 $nfiles) 2>/dev/null || {
- i=1
- while test $i -le $nfiles; do
- echo $i
- i=\`expr $i + 1\`
- done; }`
+list=`seq_ 1 $nfiles`
sed "s|@limit@|$limit|g" >myinstall.in <<'END'
#! /bin/sh
END
for n in $list; do
- cat >>Makefile.am <<END
-python_PYTHON += python$n.py
-nobase_python_PYTHON += npython$n.py
+ unindent >>Makefile.am <<END
+ python_PYTHON += python$n.py
+ nobase_python_PYTHON += npython$n.py
END
echo >python$n.py
echo >npython$n.py
for file in python3.py python$nfiles.py
do
chmod a-r $srcdir/$file
- test ! -r $srcdir/$file || Exit 77
+ test ! -r $srcdir/$file || skip_ "cannot drop file read permissions"
$MAKE install && Exit 1
chmod u+r $srcdir/$file
done
limit=2500
subdir=long_subdir_name_with_many_characters
nfiles=81
-
-# Let's use `seq' if available, it's faster than the loop.
-list=`(seq 1 $nfiles) 2>/dev/null || {
- i=1
- while test $i -le $nfiles; do
- echo $i
- i=\`expr $i + 1\`
- done; }`
+list=`seq_ 1 $nfiles`
sed "s|@limit@|$limit|g" >myinstall.in <<'END'
#! /bin/sh
END
for n in $list; do
- cat >>Makefile.am <<END
-bin_SCRIPTS += script$n
-nobase_bin_SCRIPTS += nscript$n
-data_DATA += data$n
-nobase_data_DATA += ndata$n
-include_HEADERS += header$n.h
-nobase_include_HEADERS += nheader$n.h
+ unindent >>Makefile.am <<END
+ bin_SCRIPTS += script$n
+ nobase_bin_SCRIPTS += nscript$n
+ data_DATA += data$n
+ nobase_data_DATA += ndata$n
+ include_HEADERS += header$n.h
+ nobase_include_HEADERS += nheader$n.h
END
echo >script$n
echo >nscript$n
for file in script3 script$nfiles
do
chmod a-r $srcdir/$file
- test ! -r $srcdir/$file || Exit 77
+ test ! -r $srcdir/$file || skip_ "cannot drop file read permissions"
$MAKE install-binSCRIPTS && Exit 1
chmod u+r $srcdir/$file
done
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Various install-sh checks
+# Various install-sh checks.
. ./defs || Exit 1
-# Basic errors
+
+get_shell_script install-sh
+
+# Basic errors.
./install-sh && Exit 1
./install-sh -m 644 dest && Exit 1
-# Directories
+# Directories.
# It should be OK to create no directory. We sometimes need
# this when directory are conditionally defined.
test -d d2
test -d d3
test -d d4
-# Subdirectories
+# Subdirectories.
./install-sh -d p1/p2/p3 p4//p5//p6//
test -d p1/p2/p3
test -d p4/p5/p6
./install-sh -m 644 y z
test -f y
test -f z
-# Multiple files
+# Multiple files.
./install-sh -m 644 -c x z d1
test -f x
test -f z
required=non-root
. ./defs || Exit 1
+
# Solaris /usr/ucb/touch does not accept -t.
-touch -t $old_timestamp foo || Exit 77
+touch -t $old_timestamp foo \
+ || skip_ "touch utility doesn't accept '-t' option"
+
+get_shell_script install-sh
./install-sh -d d1
+++ /dev/null
-#! /bin/sh
-# Copyright (C) 2010, 2011 Free Software Foundation, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2, or (at your option)
-# any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program. If not, see <http://www.gnu.org/licenses/>.
-
-#
-# Driver script to generate and run tests checking that building from,
-# or installing to, directories with shell metacharacters succeed.
-#
-# Original report from James Amundson about file names with spaces.
-# Other characters added by Paul Eggert.
-#
-# This script fulfills a threefold role:
-# 1. It generates a Makefile.am snippet, containing the definition
-# of proper lists of tests.
-# 2. It sets up a directory containing some common data files and
-# autotools-generated files used by said generated tests (this
-# is done for speed reasons only).
-# 3. It is sourced by said generated tests with proper parameters
-# pre-set, to run the "meat" of the checks.
-# This setup might seem a tricky and over-engineered abuse, but past
-# (painful) experiences showed that it is indeed required, because
-# the test generation code and test execution code tend to be
-# inextricably coupled and intertwined.
-#
-
-# Be more Bourne compatible (snippet copied from `tests/defs').
-DUALCASE=1; export DUALCASE # for MKS sh
-if test -n "${ZSH_VERSION+set}" && (emulate sh) >/dev/null 2>&1; then
- emulate sh
- NULLCMD=:
- # Pre-4.2 versions of Zsh do word splitting on ${1+"$@"}, which
- # is contrary to our usage. Disable this feature.
- alias -g '${1+"$@"}'='"$@"'
- setopt NO_GLOB_SUBST
-else
- case `(set -o) 2>/dev/null` in *posix*) set -o posix;; esac
-fi
-
-set -e
-
-# Sanity and usage checks.
-if test x"$instspc_action" = x; then
- if test "$#,$1" = "1,--generate-makefile"; then
- instspc_action=generate-makefile
- else
- echo "$0: empty action and no proper command line" >&2
- exit 99
- fi
-elif test $# -gt 0; then
- echo "$0: action specified and command line arguments used" >&2
- exit 99
-fi
-
-case $instspc_action in
- generate-makefile|generate-data)
- ;;
- test-build|test-install)
- if test x"$instspc_test_name" = x; then
- echo "$0: test name undefined for action '$instspc_action'" >&2
- exit 99
- fi
- ;;
- *)
- echo "$0: invalid action: '$instspc_action'"
- exit 99
- ;;
-esac
-
-# Helper subroutine for test data definition.
-# Usage: define_problematic_string NAME STRING
-define_problematic_string ()
-{
- tst=$1
- shift
- eval "instspc__$tst=\$1" || exit 99
- shift
- instspc_names_list="$instspc_names_list $tst"
- # Some of the "problematic" characters cannot be used in the name of
- # a build or install directory on a POSIX host. These lists should
- # be empty, but are not due to limitations in Autoconf, Automake, Make,
- # M4, or the shell.
- case " $* " in *' fail-build '*|*' build-fail '*)
- instspc_xfail_builds_list="$instspc_xfail_builds_list $tst";;
- esac
- case " $* " in *' fail-install '*|*' install-fail '*)
- instspc_xfail_installs_list="$instspc_xfail_installs_list $tst";;
- esac
-}
-
-# Helper subroutines for creation of input data files.
-
-create_input_data ()
-{
- mkdir sub
-
- unindent > configure.in << 'EOF'
- AC_INIT([instspc], [1.0])
- AM_INIT_AUTOMAKE
- AC_CONFIG_FILES([Makefile])
- AC_PROG_CC
- AC_PROG_RANLIB
- AC_OUTPUT
-EOF
-
- : > sub/base.h
- : > sub/nobase.h
- : > sub/base.dat
- : > sub/nobase.dat
- : > sub/base.sh
- : > sub/nobase.sh
-
- unindent > source.c << 'EOF'
- int
- main (int argc, char **argv)
- {
- return 0;
- }
-EOF
-
- unindent > Makefile.am << 'EOF'
- foodir = $(prefix)/foo
- fooexecdir = $(prefix)/foo
-
- foo_HEADERS = sub/base.h
- nobase_foo_HEADERS = sub/nobase.h
-
- dist_foo_DATA = sub/base.dat
- nobase_dist_foo_DATA = sub/nobase.dat
-
- dist_fooexec_SCRIPTS = sub/base.sh
- nobase_dist_fooexec_SCRIPTS = sub/nobase.sh
-
- fooexec_PROGRAMS = sub/base
- nobase_fooexec_PROGRAMS = sub/nobase
- sub_base_SOURCES = source.c
- sub_nobase_SOURCES = source.c
-
- fooexec_LIBRARIES = sub/libbase.a
- nobase_fooexec_LIBRARIES = sub/libnobase.a
- sub_libbase_a_SOURCES = source.c
- sub_libnobase_a_SOURCES = source.c
-
- .PHONY: test-install-sep
- test-install-sep: install
- test -f '$(DESTDIR)/$(file)-prefix/foo/sub/nobase.h'
- test ! -f '$(DESTDIR)/$(file)-prefix/foo/nobase.h'
- test -f '$(DESTDIR)/$(file)-prefix/foo/base.h'
- test -f '$(DESTDIR)/$(file)-prefix/foo/sub/nobase.dat'
- test ! -f '$(DESTDIR)/$(file)-prefix/foo/nobase.dat'
- test -f '$(DESTDIR)/$(file)-prefix/foo/base.dat'
- test -f '$(DESTDIR)/$(file)-prefix/foo/sub/nobase.sh'
- test ! -f '$(DESTDIR)/$(file)-prefix/foo/nobase.sh'
- test -f '$(DESTDIR)/$(file)-prefix/foo/base.sh'
- test -f '$(DESTDIR)/$(file)-prefix/foo/sub/nobase$(EXEEXT)'
- test ! -f '$(DESTDIR)/$(file)-prefix/foo/nobase$(EXEEXT)'
- test -f '$(DESTDIR)/$(file)-prefix/foo/base$(EXEEXT)'
- test -f '$(DESTDIR)/$(file)-prefix/foo/sub/libnobase.a'
- test ! -f '$(DESTDIR)/$(file)-prefix/foo/libnobase.a'
- test -f '$(DESTDIR)/$(file)-prefix/foo/libbase.a'
-EOF
-
- $ACLOCAL
- $AUTOCONF
- $AUTOMAKE -a
-
- : > success
-}
-
-# Be sure to avoid interferences from the environment.
-instspc_names_list=''
-instspc_xfail_builds_list=''
-instspc_xfail_installs_list=''
-
-
-# ================= #
-# Test data begin #
-# ----------------- #
-
-# Some control characters that are white space.
-bs='\b' # back space
-cr='\r' # carriage return
-ff='\f' # form feed
-ht=' ' # horizontal tab
-lf='
-' # line feed (aka newline)
-
-# Hack to save typing and make code visually clearer.
-def=define_problematic_string
-
-$def squote \' fail-build fail-install
-$def dquote '"' fail-build fail-install
-$def bquote '`' fail-build fail-install
-$def sharp '#' fail-build fail-install
-$def dollar '$' fail-build fail-install
-$def bang '!'
-$def bslash '\' fail-build
-$def ampersand '&' fail-build
-$def percent '%'
-$def leftpar '('
-$def rightpar ')'
-$def pipe '|'
-$def caret '^'
-$def tilde '~'
-$def qmark '?'
-$def star '*'
-$def plus '+'
-$def minus '-'
-$def comma ','
-$def colon ':'
-$def semicol ';'
-$def equal '='
-$def less '<'
-$def more '>'
-$def at '@'
-$def lqbrack '['
-$def rqbrack ']'
-$def lcbrack '{'
-$def rcbrack '}'
-$def space ' '
-$def tab "$ht"
-$def linefeed "$lf" fail-build fail-install
-$def backspace "$bs"
-$def formfeed "$ff"
-$def carriageret "$cr"
-$def quadrigraph0 '@&t@' fail-build
-$def quadrigraph1 '@<:@'
-$def quadrigraph2 '@:>@'
-$def quadrigraph3 '@S|@'
-$def quadrigraph4 '@%:@'
-$def a_b 'a b'
-$def a__b 'a b'
-$def a_lf_b "a${lf}b" fail-build fail-install
-$def dotdotdot '...'
-$def dosdrive 'a:'
-$def miscglob1 '?[a-z]*'
-$def miscglob2 '.*?[0-9]'
-
-unset def
-
-# --------------- #
-# Test data end #
-# =============== #
-
-
-if test x"$instspc_action" = x"generate-makefile"; then
- # We must generate a makefile fragment on stdout. It must refer
- # to all tests at once, hence the loop below.
- echo '## Generated by instspc-tests.sh. DO NOT EDIT!'
- echo 'instspc_tests ='
- echo 'instspc_xfail_tests ='
- for test_name in $instspc_names_list; do
- echo "instspc_tests += instspc-$test_name-build.test"
- echo "instspc_tests += instspc-$test_name-install.test"
- done
- for test_name in $instspc_xfail_builds_list; do
- echo "instspc_xfail_tests += instspc-$test_name-build.test"
- done
- for test_name in $instspc_xfail_installs_list; do
- echo "instspc_xfail_tests += instspc-$test_name-install.test"
- done
- exit 0
-fi
-
-# We'll need the full setup provided by `tests/defs'. Temporarily disable
-# the errexit flag, since the setup code might not be prepared to deal
-# with it.
-set +e
-. ./defs || Exit 99
-set -e
-
-# The directory set up by the `generate-data' action should contain all
-# the files we need. So remove the other files created by ./defs. And
-# check we really are in a temporary `*.dir' directory in the build tree,
-# since the last thing we want is to remove some random user files!
-test -f ../defs-static && test -f ../defs || Exit 99
-case `pwd` in *.dir);; *) Exit 99;; esac
-rm -f *
-
-if test x"$instspc_action" = x"generate-data"; then
- # We must *not* remove the test directory, since its contents must be
- # used by following dependent tests.
- keep_testdirs=yes
- create_input_data
- Exit 0
-fi
-
-### If we are still here, we have to run a test ...
-
-eval "instspc_test_string=\${instspc__$instspc_test_name}" || Exit 99
-if test x"$instspc_test_string" = x; then
- echo "$me: invalid test name: '$instspc_test_name'" >&2
- Exit 99
-fi
-
-test -f ../instspc-data.dir/success || {
- echo "$me: setup by instspc-data.test failed" >&2
- Exit 99
-}
-
-# Skip if this system doesn't support these characters in file names.
-mkdir "./$instspc_test_string" || Exit 77
-
-case $instspc_action in
- test-build)
- dest=`pwd`/_dest
- relbuilddir=../..
- cd "./$instspc_test_string"
- ;;
- test-install)
- dest=`pwd`/$instspc_test_string
- relbuilddir=..
- ;;
- *)
- echo "$me: internal error: invalid action '$instspc_action'"
- Exit 99
- ;;
-esac
-
-$relbuilddir/instspc-data.dir/configure \
- --prefix "/$instspc_test_string-prefix"
-$MAKE
-# Some make implementations eliminate leading and trailing whitespace
-# from macros passed on the command line, and some eliminate leading
-# whitespace from macros set from environment variables, so prepend
-# './' and use the latter here.
-# Tru64 sh -e needs '|| Exit' in order to work correctly.
-DESTDIR="$dest" file="./$instspc_test_string" $MAKE -e test-install-sep \
- || Exit 1
-
-:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2010, 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check that building from, or installing to, directories with shell
+# metacharacters succeed.
+# Original report from James Amundson about file names with spaces.
+# Other characters added by Paul Eggert.
+
+. ./defs || Exit 99
+
+plan_ 94 # Two tests per "problematic string".
+
+# Usage: is_in_list ITEM [LIST...]
+is_in_list ()
+{
+ item=$1; shift;
+ case " $* " in
+ *[\ \ ]"$item"[\ \ ]*) return 0;;
+ *) return 1;;
+ esac
+}
+
+# Helper subroutine for test data definition.
+# Usage: define_problematic_string NAME STRING
+define_problematic_string ()
+{
+ tst=$1; shift
+ eval "instspc__$tst=\$1" \
+ || fatal_ "define_problematic_string: bad argument: '$tst'"
+ shift
+ test_names_list="$test_names_list $tst"
+ # Some of the "problematic" characters cannot be used in the name of
+ # a build or install directory on a POSIX host. These lists should
+ # be empty, but are not due to limitations in Autoconf, Automake, Make,
+ # M4, or the shell.
+ if is_in_list fail-builddir "$@"; then
+ builddir_xfails="$builddir_xfails $tst"
+ fi
+ if is_in_list fail-destdir "$@"; then
+ destdir_xfails="$destdir_xfails $tst"
+ fi
+}
+
+# Be sure to avoid interferences from the environment.
+test_names_list=''
+builddir_xfails=''
+destdir_xfails=''
+
+expected_to_fail ()
+{
+ case $1 in
+ build) is_in_list "$2" $builddir_xfails;;
+ dest) is_in_list "$2" $destdir_xfails;;
+ *) fatal_ "incorrect 'expected_to_fail' usage";;
+ esac
+}
+
+# Helper subroutines for creation of input data files.
+create_input_data ()
+{
+ mkdir sub
+
+ unindent >> configure.in << 'EOF'
+ AC_PROG_CC
+ AC_PROG_RANLIB
+ AC_OUTPUT
+EOF
+
+ : > sub/base.h
+ : > sub/nobase.h
+ : > sub/base.dat
+ : > sub/nobase.dat
+ : > sub/base.sh
+ : > sub/nobase.sh
+
+ unindent > source.c << 'EOF'
+ int
+ main (int argc, char **argv)
+ {
+ return 0;
+ }
+EOF
+
+ unindent > Makefile.am << 'EOF'
+ foodir = $(prefix)/foo
+ fooexecdir = $(prefix)/foo
+
+ foo_HEADERS = sub/base.h
+ nobase_foo_HEADERS = sub/nobase.h
+
+ dist_foo_DATA = sub/base.dat
+ nobase_dist_foo_DATA = sub/nobase.dat
+
+ dist_fooexec_SCRIPTS = sub/base.sh
+ nobase_dist_fooexec_SCRIPTS = sub/nobase.sh
+
+ fooexec_PROGRAMS = sub/base
+ nobase_fooexec_PROGRAMS = sub/nobase
+ sub_base_SOURCES = source.c
+ sub_nobase_SOURCES = source.c
+
+ fooexec_LIBRARIES = sub/libbase.a
+ nobase_fooexec_LIBRARIES = sub/libnobase.a
+ sub_libbase_a_SOURCES = source.c
+ sub_libnobase_a_SOURCES = source.c
+
+ .PHONY: test-inst
+ test-inst: install
+ test -f '$(DESTDIR)/$(file)-prefix/foo/sub/nobase.h'
+ test ! -f '$(DESTDIR)/$(file)-prefix/foo/nobase.h'
+ test -f '$(DESTDIR)/$(file)-prefix/foo/base.h'
+ test -f '$(DESTDIR)/$(file)-prefix/foo/sub/nobase.dat'
+ test ! -f '$(DESTDIR)/$(file)-prefix/foo/nobase.dat'
+ test -f '$(DESTDIR)/$(file)-prefix/foo/base.dat'
+ test -f '$(DESTDIR)/$(file)-prefix/foo/sub/nobase.sh'
+ test ! -f '$(DESTDIR)/$(file)-prefix/foo/nobase.sh'
+ test -f '$(DESTDIR)/$(file)-prefix/foo/base.sh'
+ test -f '$(DESTDIR)/$(file)-prefix/foo/sub/nobase$(EXEEXT)'
+ test ! -f '$(DESTDIR)/$(file)-prefix/foo/nobase$(EXEEXT)'
+ test -f '$(DESTDIR)/$(file)-prefix/foo/base$(EXEEXT)'
+ test -f '$(DESTDIR)/$(file)-prefix/foo/sub/libnobase.a'
+ test ! -f '$(DESTDIR)/$(file)-prefix/foo/libnobase.a'
+ test -f '$(DESTDIR)/$(file)-prefix/foo/libbase.a'
+EOF
+
+ $ACLOCAL || framework_failure_ "aclocal failed"
+ $AUTOCONF || framework_failure_ "autoconf failed"
+ $AUTOMAKE || framework_failure_ "automake failed"
+}
+
+# ================= #
+# Test data begin #
+# ----------------- #
+
+# Some control characters that are white space.
+bs='\b' # back space
+cr='\r' # carriage return
+ff='\f' # form feed
+ht=' ' # horizontal tab
+lf='
+' # line feed (aka newline)
+
+# Hack to save typing and make code visually clearer.
+def=define_problematic_string
+
+$def squote \' fail-builddir fail-destdir
+$def dquote '"' fail-builddir fail-destdir
+$def bquote '`' fail-builddir fail-destdir
+$def sharp '#' fail-builddir fail-destdir
+$def dollar '$' fail-builddir fail-destdir
+$def bang '!'
+$def bslash '\' fail-builddir
+$def ampersand '&' fail-builddir
+$def percent '%'
+$def leftpar '('
+$def rightpar ')'
+$def pipe '|'
+$def caret '^'
+$def tilde '~'
+$def qmark '?'
+$def star '*'
+$def plus '+'
+$def minus '-'
+$def comma ','
+$def colon ':'
+$def semicol ';'
+$def equal '='
+$def less '<'
+$def more '>'
+$def at '@'
+$def lqbrack '['
+$def rqbrack ']'
+$def lcbrack '{'
+$def rcbrack '}'
+$def space ' '
+$def tab "$ht"
+$def linefeed "$lf" fail-builddir fail-destdir
+$def backspace "$bs"
+$def formfeed "$ff"
+$def carriageret "$cr"
+$def quadrigraph0 '@&t@' fail-builddir
+$def quadrigraph1 '@<:@'
+$def quadrigraph2 '@:>@'
+$def quadrigraph3 '@S|@'
+$def quadrigraph4 '@%:@'
+$def a_b 'a b'
+$def a__b 'a b'
+$def a_lf_b "a${lf}b" fail-builddir fail-destdir
+$def dotdotdot '...'
+$def dosdrive 'a:'
+$def miscglob1 '?[a-z]*'
+$def miscglob2 '.*?[0-9]'
+
+unset def
+
+# --------------- #
+# Test data end #
+# =============== #
+
+ocwd=`pwd` || fatal_ "cannot get current working directory"
+
+create_input_data
+
+for test_name in $test_names_list; do
+
+ eval "test_string=\${instspc__$test_name}" \
+ && test x"$test_string" != x \
+ || fatal_ "invalid test name: '$test_name'"
+
+ # Skip the next checks if this system doesn't support the required
+ # characters in file names.
+
+ mkdir "./$test_string" || {
+ skip_ -r "mkdir failed" "$test_name in builddir"
+ skip_ -r "mkdir failed" "$test_name in destdir"
+ continue
+ }
+
+ # Where are the "weird" characters going to be used, in $(builddir)
+ # or in $(DESTDIR)? They are always going to be used in $(prefix)
+ # though; should we maybe separate this into a dedicated check?
+ for where in build dest; do
+
+ case $where in
+ build)
+ build=./$test_string
+ dest=$ocwd/dest-$test_name
+ ;;
+ dest)
+ build=build-$test_name
+ dest=$ocwd/$test_string
+ mkdir "$build" || fatal_ "cannot create '$build'"
+ ;;
+ *)
+ fatal_ "invalid where '$where'"
+ ;;
+ esac
+
+ cd "$build" || fatal_ "cannot chdir into '$build'"
+
+ # Some make implementations eliminate leading and trailing whitespace
+ # from macros passed on the command line, and some eliminate leading
+ # whitespace from macros set from environment variables, so prepend
+ # './' and use the latter here.
+ r=ok
+ ../configure --prefix "/$test_string-prefix" \
+ && $MAKE all \
+ && DESTDIR="$dest" file="./$test_string" $MAKE -e test-inst \
+ || r='not ok'
+
+ description="$test_name in ${where}dir"
+ if expected_to_fail "$where" "$test_name"; then
+ directive=TODO
+ reason="long-standing limitation"
+ else
+ directive=
+ reason=
+ fi
+ # Test case outcome is here.
+ result_ "$r" -D "$directive" -r "$reason" -- "$description"
+
+ cd "$ocwd" || fatal_ "cannot chdir back to test directory"
+
+ # Remove subdirectories for tests that have passed, to avoid ending up
+ # with a too big test directory. This is especially important since
+ # some tests in this tests are expected to fail, and this will cause
+ # the test directory not to be removed when the script terminates.
+ case $keep_testdirs,$r1 in
+ ,ok) rm_rf_ $depmode-$vpath.d;;
+ *) : For lesser shells with broken 'set -e';;
+ esac
+
+ done # $instspc_action
+
+done # $test_name
+
+:
# "make check" should compile files in $(check_JAVA) ...
$MAKE check
-ls -l # for debugging
+ls -l # For debugging.
test -f One.class
test -f Two.class
# ... but should *not* install them.
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check that we can get use the `yywrap' function from a system-wide
+# library, if that's available.
+
+required='cc lex'
+. ./defs || Exit 1
+
+cat >> configure.in << 'END'
+AC_PROG_CC
+AC_PROG_RANLIB
+AC_PROG_LEX
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+bin_PROGRAMS = lexer
+lexer_SOURCES = foo.l
+lexer_LDADD = $(LEXLIB)
+
+.PHONY: have-lexlib
+have-lexlib:
+ test x'$(LEXLIB)' != x
+ echo 'int main (void) { return yywrap (); }' > x.c
+ $(CC) -c x.c
+ $(CC) x.$(OBJEXT) $(LEXLIB)
+ rm -f x.c *.$(OBJEXT) *.o *.out *.exe
+END
+
+cat > foo.l <<'END'
+%%
+"GOOD" return EOF;
+.
+%%
+int main (void)
+{
+ /* We don't use a 'while' loop here (like a real lexer would do)
+ to avoid possible hangs. */
+ if (yylex () == EOF)
+ return 0;
+ else
+ return 1;
+}
+END
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+./configure
+$MAKE have-lexlib || skip_ "no system-wide lex library found"
+
+# Program should build and run and distribute.
+$MAKE all
+if cross_compiling; then :; else
+ echo GOOD | ./lexer
+ echo BAD | ./lexer && Exit 1
+fi
+$MAKE distcheck
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 1999, 2001, 2002, 2003, 2004, 2010, 2011 Free Software
+# Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check that we can provide a personal `yywrap' function in a custom
+# library.
+# See also test `lex-lib-external.test'.
+
+required='cc lex'
+. ./defs || Exit 1
+
+cat >> configure.in << 'END'
+AC_PROG_CC
+AC_PROG_RANLIB
+LEXLIB=libmylex.a
+AC_PROG_LEX
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+bin_PROGRAMS = lexer
+lexer_SOURCES = foo.l
+lexer_LDADD = $(LEXLIB)
+EXTRA_lexer_DEPENDENCIES = $(LEXLIB)
+noinst_LIBRARIES = libmylex.a
+libmylex_a_SOURCES = mu.c
+END
+
+cat > mu.c << 'END'
+int yywrap (void)
+{
+ return 1;
+}
+END
+
+cat > foo.l <<'END'
+%%
+"END" return EOF;
+.
+%%
+int main (void)
+{
+ return 0;
+}
+END
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+./configure
+$MAKE
+test -f foo.c
+test -f libmylex.a
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 1999, 2001, 2002, 2003, 2004, 2010, 2011 Free Software
+# Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check that we can provide a personal `yywrap' function through the
+# LIBOBJ machinery.
+
+required='cc lex'
+. ./defs || Exit 1
+
+cat >> configure.in << 'END'
+AC_PROG_CC
+AC_PROG_RANLIB
+AC_PROG_LEX
+save_LIBS=$LIBS
+LIBS="$LEXLIB $LIBS"
+AC_REPLACE_FUNCS([yywrap])
+LIBS=$save_LIBS
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+noinst_PROGRAMS = foo
+foo_SOURCES = foo.l
+foo_LDADD = $(LEXLIB) $(LIBOBJS)
+END
+
+cat > yywrap.c << 'END'
+int yywrap (void)
+{
+ return 1;
+}
+END
+
+cat > foo.l <<'END'
+%%
+"END" return EOF;
+.
+%%
+int main (void)
+{
+ return 0;
+}
+END
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+./configure
+grep LIBOBJS Makefile # For debugging.
+$MAKE
+$MAKE distclean
+
+# Force "no system lex library".
+./configure LEXLIB='-L /lib'
+grep LIBOBJS Makefile # For debugging.
+grep '^LIBOBJS *=.*yywrap.*\.o' Makefile # Sanity check.
+$MAKE
+
+$MAKE distcheck
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 1999, 2001, 2002, 2003, 2004, 2010, 2011 Free Software
+# Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check Lex support with flex using the `%noyywrap' option.
+
+required='cc flex'
+. ./defs || Exit 1
+
+cat >> configure.in << 'END'
+AC_PROG_CC
+AM_PROG_LEX
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+bin_PROGRAMS = foo
+foo_SOURCES = foo.l
+
+.PHONY: test-no-lexlib
+check-local: test-no-lexlib
+test-no-lexlib:
+ test x'$(LEXLIB)' = x'none needed'
+END
+
+cat > foo.l << 'END'
+%option noyywrap
+%%
+"GOOD" return EOF;
+.
+%%
+int main (void)
+{
+ /* We don't use a 'while' loop here (like a real lexer would do)
+ to avoid possible hangs. */
+ if (yylex () == EOF)
+ return 0;
+ else
+ return 1;
+}
+END
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+./configure LEXLIB="none needed"
+
+# Program should build and run.
+$MAKE
+if cross_compiling; then :; else
+ echo GOOD | ./foo
+ echo BAD | ./foo && Exit 1
+fi
+
+# Sanity check on distribution.
+$MAKE distcheck DISTCHECK_CONFIGURE_FLAGS='LEXLIB="none needed"'
+
+:
# Ensure subdirs for subdir scanners are generated when subdir-objects
# are used, even when dependency tracking is disabled.
-required=flex
+required='cc flex'
. ./defs || Exit 1
cat >>configure.in <<\END
int yywrap(void)
{
- return 0;
+ return 1;
}
END
# Test associated with PR 19.
# From Matthew D. Langston.
-required='gcc flex GNUmake'
+required='cc flex'
. ./defs || Exit 1
-# Ignore user CFLAGS.
-CFLAGS=
-export CFLAGS
-
cat >> configure.in << 'END'
AC_PROG_CC
AM_PROG_LEX
END
cat > Makefile.am << 'END'
-LDADD = @LEXLIB@
-noinst_PROGRAMS = foo
-foo_SOURCES = foo.l
+noinst_PROGRAMS = foo
+foo_SOURCES = foo.l
END
cat > foo.l << 'END'
"GOOD" return EOF;
.
%%
-int
-main ()
+
+int main (void)
{
/* We don't use a 'while' loop here (like a real lexer would do)
to avoid possible hangs. */
else
return 1;
}
+
+/* Avoid possible link errors. */
+int yywrap (void)
+{
+ return 1;
+}
END
$ACLOCAL
# Program should build and run.
$MAKE
-echo GOOD | ./foo
-echo BAD | ./foo && Exit 1
+if cross_compiling; then :; else
+ echo GOOD | ./foo
+ echo BAD | ./foo && Exit 1
+fi
# The generated file `foo.c' must be shipped.
$MAKE distdir
test -f foo.c
$MAKE distclean
test -f foo.c
-./configure # we must re-create `Makefile'
+./configure # Re-create `Makefile'.
$MAKE maintainer-clean
test ! -f foo.c
# Test for subdir lexers.
-required='gcc GNUmake flex'
+required='cc flex'
. ./defs || Exit 1
cat >> configure.in << 'END'
test -f ./ylwrap
cd sub
+using_gmake || $MAKE Makefile
$MAKE foo/foo2.o
test -f foo/foo2.c
test -f foo/foo2.o
# Please keep this in sync with sister test `yaccvapth.test'.
-required='gcc flex'
+required='cc flex'
. ./defs || Exit 1
cat > lexoutroot.in << 'END'
END
cat > foo.c << 'END'
-int main () { return 0; }
+int main (void)
+{
+ return 0;
+}
+/* Avoid possible link errors. */
+int yywrap (void)
+{
+ return 1;
+}
END
$ACLOCAL
mkdir sub
-# We must run configure early, to find out whay $LEX_OUTPUT_ROOT is.
+# We must run configure early, to find out why $LEX_OUTPUT_ROOT is.
cd sub
../configure
. ./lexoutroot
-test -n "$LEX_OUTPUT_ROOT" # sanity check
+test -n "$LEX_OUTPUT_ROOT" # Sanity check.
cd ..
flex lexer.l
# Check that $(LFLAGS) takes precedence over both $(AM_LFLAGS) and
# $(foo_LFLAGS).
# Please keep this in sync with the sister tests lflags2.test, yflags.test
-# and yflags2.test
+# and yflags2.test.
+required=cc
. ./defs || Exit 1
cat >fake-lex <<'END'
unset LEX || :
cat >> configure.in <<'END'
-AC_PROG_CC
+AC_SUBST([CC], [false])
# Simulate presence of Lex using our fake-lex script.
AC_SUBST([LEX], ['$(abs_top_srcdir)'/fake-lex])
AC_SUBST([LEX_OUTPUT_ROOT], [lex.yy])
END
cat > Makefile.am <<'END'
+AUTOMAKE_OPTIONS = no-dependencies
bin_PROGRAMS = foo bar
foo_SOURCES = main.c foo.l
bar_SOURCES = main.c bar.l
grep '\$(LFLAGS).*\$(bar_LFLAGS)' Makefile.in && Exit 1
grep '\$(LFLAGS).*\$(AM_LFLAGS)' Makefile.in && Exit 1
-echo 'int main(void){ return 0; }' > main.c
: > foo.l
: > bar.l
$AUTOCONF
./configure
-env LFLAGS=__user_flags__ $MAKE -e
-
-ls -l
+env LFLAGS=__user_flags__ $MAKE -e foo.c bar-bar.c
cat foo.c
-grep '__am_flags__.*__user_flags__' foo.c
cat bar-bar.c
+
+grep '__am_flags__.*__user_flags__' foo.c
grep '__bar_flags__.*__user_flags__' bar-bar.c
:
# Check that $(LFLAGS) takes precedence over both $(AM_LFLAGS) and
# $(foo_LFLAGS).
# Please keep this in sync with the sister tests lflags.test, yflags.test
-# and yflags2.test
+# and yflags2.test.
-required=g++ # FIXME: any working C++ compiler should be OK
. ./defs || Exit 1
cat >fake-lex <<'END'
unset LEX || :
cat >> configure.in <<'END'
-AC_PROG_CXX
+AC_SUBST([CXX], [false])
# Simulate presence of Lex using our fake-lex script.
AC_SUBST([LEX], ['$(abs_top_srcdir)'/fake-lex])
AC_SUBST([LEX_OUTPUT_ROOT], [lex.yy])
END
cat > Makefile.am <<'END'
+AUTOMAKE_OPTIONS = no-dependencies
bin_PROGRAMS = foo bar
foo_SOURCES = main.cc foo.ll
bar_SOURCES = main.cc bar.l++
grep '\$(LFLAGS).*\$(bar_LFLAGS)' Makefile.in && Exit 1
grep '\$(LFLAGS).*\$(AM_LFLAGS)' Makefile.in && Exit 1
-echo 'int main(void) { return 0; }' > main.cc
: > foo.ll
: > bar.l++
$AUTOCONF
./configure
-env LFLAGS=__user_flags__ $MAKE -e
-
-ls -l
+env LFLAGS=__user_flags__ $MAKE -e foo.cc bar-bar.c++
cat foo.cc
-grep '__am_flags__.*__user_flags__' foo.cc
cat bar-bar.c++
+
+grep '__am_flags__.*__user_flags__' foo.cc
grep '__bar_flags__.*__user_flags__' bar-bar.c++
:
# Make sure AC_LIBSOURCE and AC_LIBSOURCES work.
+required=cc
. ./defs || Exit 1
mv configure.in configure.proto
# Do not complain about the nonexistence of a source for LIBOBJS if
# it's in BUILT_SOURCES. Reported by Erez Zadok.
+required=cc
. ./defs || Exit 1
cat >> configure.in << 'END'
./configure
$MAKE
-ar t libfoo.a # for debugging
+ar t libfoo.a # For debugging.
$MAKE distcheck
:
# Regression test for an internal error when @LIBOBJS@ is used in
# a variable that is not defined in the same conditions as the _LDADD
# that uses it.
-# Report from Bill Davidson
+# Report from Bill Davidson.
-required=gcc
. ./defs || Exit 1
cat >>configure.in <<'EOF'
# Make sure AC_LIBOBJ and friends work.
# Please keep this in sync with sister test `libobj16b.test'.
+required=cc
. ./defs || Exit 1
mv configure.in configure.proto
cat >> configure.proto << 'END'
AC_PROG_CC
AC_PROG_RANLIB
-%LIBOBJ-STUFF% # will be activated later
+%LIBOBJ-STUFF% # Will be activated later.
AC_OUTPUT
END
i\
AC_LIBOBJ([maude])
}' configure.proto > configure.in
-cat configure.in # for debugging
+cat configure.in # For debugging.
cat > extra-checks.am << 'END'
extra_checks = maude-src maude-dist maude-lib liver-not-dist
fi\
AC_LIBSOURCES([maude.c, liver.c])
}' configure.proto > configure.in
-cat configure.in # for debugging
+cat configure.in # For debugging.
$ACLOCAL
$AUTOCONF
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Make sure AC_LIBOBJ and friends work.
-# Please keep this in sync with sister test `libobj16b.test'.
+# Please keep this in sync with sister test `libobj16a.test'.
+required=cc
. ./defs || Exit 1
mv configure.in configure.proto
cat >> configure.proto << 'END'
AC_PROG_CC
AC_PROG_RANLIB
-%LIBOBJ-STUFF% # will be activated later
+%LIBOBJ-STUFF% # Will be activated later.
AC_OUTPUT
END
i\
AC_LIBOBJ(maude) dnl: do not quote this!
}' configure.proto > configure.in
-cat configure.in # for debugging
+cat configure.in # For debugging.
cat > extra-checks.am << 'END'
extra_checks = maude-src maude-dist maude-lib liver-not-dist
AC_LIBSOURCE(maude.c) dnl: do not quote this!\
AC_LIBSOURCE(liver.c) dnl: do not quote this!
}' configure.proto > configure.in
-cat configure.in # for debugging
+cat configure.in # For debugging.
$ACLOCAL
$AUTOCONF
# Make sure AC_LIBOBJ accept non-literal arguments.
+required=cc
. ./defs || Exit 1
cat >> configure.in << 'END'
./configure FOO=quux
$MAKE
-ls -l # for debugging
+ls -l # For debugging.
test -f libtu.a
ar t libtu.a | $FGREP quux
ar t libtu.a | $FGREP zardoz && Exit 1
./configure FOO=zardoz
$MAKE
-ls -l # for debugging
+ls -l # For debugging.
test -f libtu.a
ar t libtu.a | $FGREP zardoz
ar t libtu.a | $FGREP quux && Exit 1
# Test support for AC_CONFIG_LIBOBJ_DIR.
+required=cc
. ./defs || Exit 1
cat >> configure.in << 'END'
# Test to make sure LIBOBJS works in subdirs.
# Bug from Josh MacDonald.
+required=cc
. ./defs || Exit 1
cat >> configure.in << 'END'
grep 'configure\.in:.*required directory.*libobj-dir' stderr
mkdir libobj-dir
-: > foobar.c # oops, it should be in libobj-dir...
+: > foobar.c # Oops, it should be in libobj-dir!
AUTOMAKE_fails
grep 'configure\.in:.*required file.*libobj-dir/foobar.c.*' stderr
rm -f foobar.c
: > libobj-dir/foobar.c
-$AUTOMAKE # now we should succeed
+$AUTOMAKE # Now we should succeed.
:
grep 'configure\.in:.*required directory.*\./libobj-dir' stderr
mkdir libobj-dir
-: > foobar.c # oops, it should be in libobj-dir!
+: > foobar.c # Oops, it should be in libobj-dir!
AUTOMAKE_fails
grep 'configure\.in:.*required file.*libobj-dir/foobar.c.*' stderr
rm -f foobar.c
: > libobj-dir/foobar.c
-$AUTOMAKE # now we should succeed
+$AUTOMAKE # Now we should succeed.
:
# Test for multiple replacement functions.
+required=cc
. ./defs || Exit 1
cat >> configure.in << 'END'
# Make sure .libs directories are removed for _PROGRAMS.
# Report from Guillermo Ontañón.
-required=libtoolize
+required='cc libtoolize'
. ./defs || Exit 1
+
cat >> configure.in << 'END'
AC_PROG_LIBTOOL
AC_OUTPUT
# Make sure config.lt is removed with Libtool 2.2.x's LT_OUTPUT.
# Report by Charles Wilson.
-required=libtoolize
+required='cc libtoolize'
. ./defs || Exit 1
+
cat >> configure.in << 'END'
AC_PROG_LIBTOOL
m4_ifdef([LT_OUTPUT], [LT_OUTPUT])
$AUTOCONF
./configure
$MAKE distcheck
+
:
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Make sure libtool is removed.
-# Report from Kevin Dalley
+# Report from Kevin Dalley.
required=libtool
. ./defs || Exit 1
# Try to build and package a program linked to a Libtool library.
# Also make sure we do not bloat the Makefile with unneeded rules.
-required='libtoolize gcc'
+required='cc libtoolize'
. ./defs || Exit 1
cat >> configure.in << 'END'
int zero ();
int
-main ()
+main (void)
{
return zero ();
}
$MAKE
$MAKE distcheck
+
+:
# Make sure we allow Libtool's -dlopen/-dlpreopen
# Also check basic support for AM_LIBTOOLFLAGS/LIBTOOLFLAGS
-required='libtoolize gcc'
+required='cc libtoolize'
. ./defs || Exit 1
cat >> configure.in << 'END'
prg_SOURCES = prg.c
prg_LDADD = -dlopen libmod1.la -dlpreopen mod2.la
+.PHONY: print
print:
@echo 1BEG: $(prg_DEPENDENCIES) :END1
@echo 2BEG: $(libmod1_la_DEPENDENCIES) :END2
mkdir sub liba
cat > sub/mod1.c << 'END'
-int
-mod1 ()
+int mod1 (void)
{
return 1;
}
END
cat > mod2.c << 'END'
-int
-mod2 ()
+int mod2 (void)
{
return 2;
}
END
cat > prg.c << 'END'
-int
-main ()
+int main (void)
{
return 0;
}
}
cat output
grep 'silent.*silent.*libmod1' output
+
+:
# Make sure xxx_LINK is defined for each target that requires specific
# flags.
-# Quite similar to libtool7.test, using AM_LDFLAGS in addition to xxx_LDFLAGS.
+# Quite similar to libtool7.test, using AM_LDFLAGS in addition to
+# xxx_LDFLAGS.
-required='libtoolize gcc'
+required='cc libtoolize'
. ./defs || Exit 1
cat >> configure.in << 'END'
mkdir liba
cat > mod1.c << 'END'
-int
-mod1 ()
+int mod1 (void)
{
return 1;
}
END
cat > mod2.c << 'END'
-int
-mod2 ()
+int mod2 (void)
{
return 2;
}
END
cat > prg.c << 'END'
-int
-main ()
+int main (void)
{
return 0;
}
grep '6BEG:.* prg2_cflags cflags .*am_ldflags ldflags.* :END6' output
grep '6BEG: .*am_cflags.* :END6' output && Exit 1
$MAKE
+
+:
# Make sure COPYING is not overwritten, even with -a -f.
-required=makeinfo
. ./defs || Exit 1
echo AC_OUTPUT >>configure.in
+
cat >Makefile.am <<\EOF
test1: distdir
grep 'GNU GENERAL PUBLIC LICENSE' $(distdir)/COPYING
./configure
$MAKE test1
-# rm before echo, because COPYING is likely to be a symlink to
+# Use 'rm' before 'echo', because COPYING is likely to be a symlink to
# the real COPYING...
rm -f COPYING
echo 'MY-OWN-LICENSE' >COPYING
AUTOMAKE_fails
grep 'COPYING' stderr
-AUTOMAKE_run 0 --add-missing
+AUTOMAKE_run --add-missing
grep 'COPYING' stderr
grep 'GNU General Public License' stderr
grep 'Consider adding.*version control' stderr
. ./defs || Exit 1
-echo 1
+
+: TRY 1 -- We lack both EMACS and lispdir.
cat > Makefile.am << 'END'
lisp_LISP = foo.el
$ACLOCAL
AUTOMAKE_fails
grep AM_PATH_LISPDIR stderr
+grep '[Ll]isp source.*EMACS.* undefined' stderr
+grep '[Ll]isp source.*lispdir.* undefined' stderr
+grep ' add .*AM_PATH_LISPDIR' stderr
-echo 2
-# Setting lispdir should not be enough.
+: TRY 2 -- Setting lispdir should not be enough.
cat > Makefile.am << 'END'
lispdir = /usr/share/emacs/site-lisp
$ACLOCAL
AUTOMAKE_fails
-grep AM_PATH_LISPDIR stderr
+grep 'lispdir.*undefined' stderr && Exit 1
+grep '[Ll]isp source.*EMACS.* undefined' stderr
+grep 'define .*EMACS.* add .*AM_PATH_LISPDIR' stderr
+
-echo 3
-# Setting EMACS should not be enough.
+: TRY 3 -- Setting EMACS should not be enough.
cat > Makefile.am << 'END'
EMACS = emacs
$ACLOCAL
AUTOMAKE_fails
-grep AM_PATH_LISPDIR stderr
+grep 'EMACS.*undefined' stderr && Exit 1
+grep '[Ll]isp source.*lispdir.* undefined' stderr
+grep 'define .*lispdir.* add .*AM_PATH_LISPDIR' stderr
-echo 4
-# Setting both is OK.
+: TRY 4 -- Setting both EMACS and lispdir is OK.
cat > Makefile.am << 'END'
lispdir = /usr/share/emacs/site-lisp
$ACLOCAL
$AUTOMAKE -a
+
+:
echo "(require 'am-three) (provide 'am-two)" > am-two.el
# am-three.el is a built source
-
$ACLOCAL
$AUTOCONF
$AUTOMAKE --add-missing
-./configure
+
+./configure --prefix="`pwd`/_inst"
$MAKE
$MAKE
test -f am-one.elc
+# Test installation/deinstallation.
+
+$MAKE install
+
+find _inst # For debugging.
+
+# Keep thin in sync with m4/lispdir.m4.
+for dir in lib/emacs lib/xemacs share/emacs share/xemacs :; do
+ if test $dir = :; then
+ Exit 1
+ elif test -d _inst/$dir/site-lisp; then
+ break
+ fi
+done
+
+test -f _inst/$dir/site-lisp/am-one.el
+test -f _inst/$dir/site-lisp/am-one.elc
+test -f _inst/$dir/site-lisp/am-two.el
+test -f _inst/$dir/site-lisp/am-two.elc
+test -f _inst/$dir/site-lisp/am-three.el
+test -f _inst/$dir/site-lisp/am-three.elc
+
+$MAKE uninstall
+find _inst | $EGREP '\.elc?$' && Exit 1
+
# Make sure we build all files when any of them change.
# (We grep a message to make sure the compilation happens.)
unique=0a3346e2af8a689b85002b53df09142a
$sleep
echo "(message \"$unique\")(provide 'am-three)" > am-three.el
-$MAKE >output 2>&1
+$MAKE >output 2>&1 || { cat output; Exit 1; }
cat output
grep $unique output
# It should also work for VPATH-builds.
$MAKE distcheck
+
+:
test ! -f "$(lispdir)/am-two.elc"
test ! -f "$(lispdir)/am-three.elc"
-install-test2: install
- test ! -f "$(lispdir)/am-one.el"
- test ! -f "$(lispdir)/am-two.el"
- test ! -f "$(lispdir)/am-three.el"
- test ! -f "$(lispdir)/am-one.elc"
- test ! -f "$(lispdir)/am-two.elc"
- test ! -f "$(lispdir)/am-three.elc"
+not-installed:
+ find "$(lispdir)" | grep '\.el$$' && exit 1; :
+ find "$(lispdir)" | grep '\.elc$$' && exit 1; :
EOF
cat >> configure.in << 'EOF'
$MAKE test
$MAKE install-test
$MAKE uninstall
+$MAKE not-installed
# Fake the absence of emacs.
-# *.el files should not be installed.
+# *.el files should not be installed, but "make install" and
+# "make uninstall" should continue to work.
./configure EMACS=no --prefix "`pwd`"
$MAKE
$MAKE test
-$MAKE install-test2
+$MAKE install
+$MAKE not-installed
+$MAKE uninstall
+
+:
test ! -f "$(lispdir)/am-one.elc"
test ! -f "$(lispdir)/am-two.elc"
test ! -f "$(lispdir)/am-three.elc"
+
+not-installed:
+ find "$(lispdir)" | grep '\.el$$' && exit 1; :
+ find "$(lispdir)" | grep '\.elc$$' && exit 1; :
EOF
cat >> configure.in << 'EOF'
$MAKE test
$MAKE install-test
$MAKE uninstall
+$MAKE not-installed
# Fake the absence of emacs.
-# *.el files SHOULD be installed.
+# *.el files SHOULD be installed by "make install" (and uninstalled
+# by "make uninstall").
./configure EMACS=no --prefix "`pwd`"
$MAKE
$MAKE test
$MAKE install-test
+$MAKE uninstall
+$MAKE not-installed
+
+:
EOF
cat >> configure.in << 'EOF'
-AM_CONDITIONAL([WANT_TWO], test -n "$want_two")
+AM_CONDITIONAL([WANT_TWO], [test -n "$want_two"])
AM_PATH_LISPDIR
AC_OUTPUT
EOF
$ACLOCAL
$AUTOCONF
$AUTOMAKE --add-missing
+
./configure "--with-lispdir=`pwd`/lisp"
$MAKE
test ! -f am-three.elc
test ! -f elc-stamp
-
./configure "--with-lispdir=`pwd`/lisp" want_two=1
+
$MAKE
test -f am-one.elc
test -f am-two.elc
test -f am-three.elc
test -f elc-stamp
-# Let's mutilate the source tree, the check the recover rule.
+# Let's mutilate the source tree, to check the recover rule.
rm -f am-*.elc
$MAKE
test -f am-one.elc
test ! -f am-two.elc
test ! -f am-three.elc
test ! -f elc-stamp
+
+:
$AUTOMAKE --add-missing
./configure
-$MAKE >stdout
-
+$MAKE >stdout || { cat stdout; Exit 1; }
cat stdout
test 1 -eq `grep 'Warnings can be ignored' stdout | wc -l`
test -f elc-stamp
$MAKE distcheck
+
+:
rm -f am-*.elc
: >stdout
-$MAKE -j >>stdout
+$MAKE -j >>stdout || { cat stdout; Exit 1; }
cat stdout
test 1 -eq `grep 'Warnings can be ignored' stdout | wc -l`
test -f am-two.elc
test -f am-three.elc
test -f elc-stamp
+
+:
--- /dev/null
+## Makefile fragment that holds the list of test scripts of the automake
+## testsuite. This fragment is meant to be included by the Makefile.am,
+## but also to be executed directly by make when bootstrapping automake.
+
+## Copyright (C) 2011 Free Software Foundation, Inc.
+##
+## This program is free software; you can redistribute it and/or modify
+## it under the terms of the GNU General Public License as published by
+## the Free Software Foundation; either version 2, or (at your option)
+## any later version.
+##
+## This program is distributed in the hope that it will be useful,
+## but WITHOUT ANY WARRANTY; without even the implied warranty of
+## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+## GNU General Public License for more details.
+##
+## You should have received a copy of the GNU General Public License
+## along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# The order here is mostly alphabetical, with the deliberate exception
+# that tests having a high runtime (especially TAP tests that run various
+# checks sequentially) are listed early; this improved performance on
+# concurrent testsuite runs.
+handwritten_TESTS = \
+depmod.tap \
+instspc.tap \
+aclocal.test \
+aclocal3.test \
+aclocal4.test \
+aclocal5.test \
+aclocal6.test \
+aclocal7.test \
+aclocal8.test \
+aclocal9.test \
+acloca10.test \
+acloca11.test \
+acloca12.test \
+acloca13.test \
+acloca14.test \
+acloca15.test \
+acloca16.test \
+acloca17.test \
+acloca18.test \
+acloca19.test \
+acloca20.test \
+acloca21.test \
+acloca22.test \
+acloca23.test \
+ac-output-old.tap \
+acsilent.test \
+acsubst.test \
+acsubst2.test \
+add-missing.tap \
+all.test \
+all2.test \
+alloca.test \
+alloca2.test \
+alpha.test \
+alpha2.test \
+amhello-cflags.test \
+amhello-cross-compile.test \
+amhello-binpkg.test \
+amassign.test \
+ammissing.test \
+amopt.test \
+amopts-location.test \
+amopts-variable-expansion.test \
+amsubst.test \
+ansi2knr-no-more.test \
+ar-lib.test \
+ar.test \
+ar2.test \
+asm.test \
+asm2.test \
+asm3.test \
+autodist.test \
+autodist-subdir.test \
+autodist-acconfig.test \
+autodist-acconfig-no-subdir.test \
+autodist-aclocal-m4.test \
+autodist-config-headers.test \
+autodist-configure-no-subdir.test \
+autodist-no-duplicate.test \
+autodist-stamp-vti.test \
+autohdr.test \
+autohdr2.test \
+autohdr3.test \
+autohdr4.test \
+autohdrdry.test \
+automake-cmdline.tap \
+auxdir.test \
+auxdir6.test \
+auxdir7.test \
+auxdir8.test \
+auxdir-autodetect.test \
+auxdir-computed.tap \
+auxdir-misplaced.test \
+auxdir-nonexistent.test \
+auxdir-unportable.tap \
+backcompat.test \
+backcompat2.test \
+backcompat3.test \
+backcompat4.test \
+backcompat5.test \
+backcompat6.test \
+backsl.test \
+backsl2.test \
+backsl3.test \
+backsl4.test \
+badline.test \
+badopt.test \
+badprog.test \
+block.test \
+bsource.test \
+candist.test \
+canon.test \
+canon2.test \
+canon3.test \
+canon4.test \
+canon5.test \
+canon6.test \
+canon7.test \
+canon8.test \
+canon-name.test \
+ccnoco.test \
+ccnoco2.test \
+ccnoco3.test \
+check.test \
+check2.test \
+check3.test \
+check4.test \
+check5.test \
+check6.test \
+check7.test \
+check8.test \
+check10.test \
+check11.test \
+check12.test \
+check-subst.test \
+check-subst-prog.test \
+check-exported-srcdir.test \
+check-fd-redirect.test \
+check-tests-in-builddir.test \
+check-no-test-driver.test \
+check-concurrency-bug9245.test \
+checkall.test \
+clean.test \
+clean2.test \
+colneq.test \
+colneq2.test \
+colneq3.test \
+colon.test \
+colon2.test \
+colon3.test \
+colon4.test \
+colon5.test \
+colon6.test \
+colon7.test \
+color.test \
+color2.test \
+comment.test \
+comment2.test \
+comment3.test \
+comment4.test \
+comment5.test \
+comment6.test \
+comment7.test \
+comment8.test \
+comment9.test \
+commen10.test \
+commen11.test \
+comments-in-var-def.test \
+compile.test \
+compile2.test \
+compile3.test \
+compile4.test \
+compile5.test \
+compile6.test \
+compile_f90_c_cxx.test \
+compile_f_c_cxx.test \
+cond-basic.test \
+cond.test \
+cond2.test \
+cond3.test \
+cond4.test \
+cond5.test \
+cond6.test \
+cond7.test \
+cond8.test \
+cond9.test \
+cond10.test \
+cond11.test \
+cond13.test \
+cond14.test \
+cond15.test \
+cond16.test \
+cond17.test \
+cond18.test \
+cond19.test \
+cond20.test \
+cond21.test \
+cond22.test \
+cond23.test \
+cond24.test \
+cond25.test \
+cond26.test \
+cond27.test \
+cond28.test \
+cond29.test \
+cond30.test \
+cond31.test \
+cond32.test \
+cond33.test \
+cond34.test \
+cond35.test \
+cond36.test \
+cond37.test \
+cond38.test \
+cond39.test \
+cond40.test \
+cond41.test \
+cond42.test \
+cond43.test \
+cond44.test \
+cond45.test \
+cond46.test \
+condd.test \
+condhook.test \
+condhook2.test \
+condinc.test \
+condinc2.test \
+condlib.test \
+condman2.test \
+condman3.test \
+configure.test \
+confdeps.test \
+conff.test \
+conff2.test \
+confh.test \
+confh4.test \
+confh5.test \
+confh6.test \
+confh7.test \
+confh8.test \
+confincl.test \
+conflnk.test \
+conflnk2.test \
+conflnk3.test \
+conflnk4.test \
+confsub.test \
+confvar.test \
+confvar2.test \
+copy.test \
+cscope.test \
+cscope2.test \
+cscope3.test \
+cxx.test \
+cxx2.test \
+cxxcpp.test \
+cxxlibobj.test \
+cxxlink.test \
+cxxnoc.test \
+cxxo.test \
+cygnus-check-without-all.test \
+cygnus-dependency-tracking.test \
+cygnus-imply-foreign.test \
+cygnus-no-dist.test \
+cygnus-no-installinfo.test \
+cygnus-requires-maintainer-mode.test \
+cygwin32.test \
+dash.test \
+defun.test \
+defun2.test \
+dejagnu.test \
+dejagnu2.test \
+dejagnu3.test \
+dejagnu4.test \
+dejagnu5.test \
+dejagnu6.test \
+dejagnu7.test \
+depacl2.test \
+depcomp.test \
+depcomp2.test \
+depcomp3.test \
+depcomp4.test \
+depcomp5.test \
+depcomp6.test \
+depcomp7.test \
+depcomp8a.test \
+depcomp8b.test \
+depdist.test \
+depend.test \
+depend2.test \
+depend3.test \
+depend4.test \
+depend5.test \
+depend6.test \
+deprecated-acinit.test \
+destdir.test \
+dirlist.test \
+dirlist2.test \
+discover.test \
+dist-included-parent-dir.test \
+distcleancheck.test \
+distcom2.test \
+distcom3.test \
+distcom4.test \
+distcom5.test \
+distcom6.test \
+distcom7.test \
+distdir.test \
+distlinks.test \
+distlinksbrk.test \
+distname.test \
+distcheck-configure-flags.test \
+distcheck-configure-flags-am.test \
+distcheck-configure-flags-subpkg.test \
+distcheck-hook.test \
+distcheck-hook2.test \
+dmalloc.test \
+dollar.test \
+dollarvar.test \
+dollarvar2.test \
+double.test \
+dup2.test \
+else.test \
+empty.test \
+empty2.test \
+empty3.test \
+empty4.test \
+exdir.test \
+exdir2.test \
+exdir3.test \
+exeext.test \
+exeext2.test \
+exeext3.test \
+exeext4.test \
+exsource.test \
+ext.test \
+ext2.test \
+ext3.test \
+extra.test \
+extra2.test \
+extra3.test \
+extra4.test \
+extra5.test \
+extra6.test \
+extra7.test \
+extra8.test \
+extra9.test \
+extra10.test \
+extra11.test \
+extra12.test \
+extra-programs-empty.test \
+extradep.test \
+extradep2.test \
+f90only.test \
+flavor.test \
+flibs.test \
+fn99.test \
+fn99subdir.test \
+fnoc.test \
+fo.test \
+forcemiss.test \
+forcemiss2.test \
+fort1.test \
+fort2.test \
+fort4.test \
+fort5.test \
+fonly.test \
+fortdep.test \
+gcj.test \
+gcj2.test \
+gcj3.test \
+gcj4.test \
+gcj5.test \
+gcj6.test \
+getopt.test \
+gettext.test \
+gettext2.test \
+gettext3.test \
+gnumake.test \
+gnuwarn.test \
+gnuwarn2.test \
+gnits.test \
+gnits2.test \
+gnits3.test \
+hdr-vars-defined-once.test \
+header.test \
+help.test \
+help2.test \
+help3.test \
+help4.test \
+help-depend.test \
+help-depend2.test \
+help-dmalloc.test \
+help-init.test \
+help-lispdir.test \
+help-maintainer.test \
+help-multilib.test \
+help-python.test \
+help-regex.test \
+help-silent.test \
+help-upc.test \
+hfs.test \
+implicit.test \
+info.test \
+init.test \
+init2.test \
+insh2.test \
+install2.test \
+installdir.test \
+instsh.test \
+instsh2.test \
+instsh3.test \
+instdat.test \
+instdat2.test \
+instdir.test \
+instdir2.test \
+instdir-java.test \
+instdir-lisp.test \
+instdir-ltlib.test \
+instdir-prog.test \
+instdir-python.test \
+instdir-texi.test \
+instexec.test \
+instfail.test \
+instfail-info.test \
+instfail-java.test \
+instfail-libtool.test \
+insthook.test \
+instman.test \
+instman2.test \
+instmany.test \
+instmany-mans.test \
+instmany-python.test \
+interp.test \
+interp2.test \
+java.test \
+java2.test \
+java3.test \
+java-check.test \
+java-empty-classpath.test \
+javaprim.test \
+javasubst.test \
+java-extra.test \
+java-noinst.test \
+ldadd.test \
+ldflags.test \
+lex.test \
+lex2.test \
+lex3.test \
+lex4.test \
+lex5.test \
+lexcpp.test \
+lexvpath.test \
+lex-subobj-nodep.test \
+lex-lib.test \
+lex-lib-external.test \
+lex-libobj.test \
+lex-noyywrap.test \
+lflags.test \
+lflags2.test \
+libexec.test \
+libobj-basic.test \
+libobj2.test \
+libobj3.test \
+libobj4.test \
+libobj5.test \
+libobj7.test \
+libobj10.test \
+libobj12.test \
+libobj13.test \
+libobj14.test \
+libobj15a.test \
+libobj15b.test \
+libobj15c.test \
+libobj16a.test \
+libobj16b.test \
+libobj17.test \
+libobj18.test \
+libobj19.test \
+libobj20a.test \
+libobj20b.test \
+libobj20c.test \
+library.test \
+library2.test \
+library3.test \
+libtool.test \
+libtool2.test \
+libtool3.test \
+libtool4.test \
+libtool5.test \
+libtool6.test \
+libtool7.test \
+libtool8.test \
+libtool9.test \
+libtoo10.test \
+libtoo11.test \
+license.test \
+license2.test \
+link_c_cxx.test \
+link_dist.test \
+link_f90_only.test \
+link_fc.test \
+link_fccxx.test \
+link_fcxx.test \
+link_f_only.test \
+link_override.test \
+lisp2.test \
+lisp3.test \
+lisp4.test \
+lisp5.test \
+lisp6.test \
+lisp7.test \
+lisp8.test \
+lispdry.test \
+listval.test \
+location.test \
+longline.test \
+longlin2.test \
+ltcond.test \
+ltcond2.test \
+ltconv.test \
+ltdeps.test \
+ltinit.test \
+ltinstloc.test \
+ltlibobjs.test \
+ltlibsrc.test \
+ltorder.test \
+lzip.test \
+lzma.test \
+m4-inclusion.test \
+maintclean.test \
+maintclean-vpath.test \
+make.test \
+makej.test \
+makej2.test \
+maken.test \
+maken2.test \
+maken3.test \
+maken4.test \
+makevars.test \
+man.test \
+man2.test \
+man3.test \
+man4.test \
+man5.test \
+man6.test \
+man7.test \
+man8.test \
+mdate.test \
+mdate2.test \
+mdate3.test \
+mdate4.test \
+mdate5.test \
+mdate6.test \
+missing.test \
+missing2.test \
+missing3.test \
+missing4.test \
+missing5.test \
+missing6.test \
+mkinstall.test \
+mkinst2.test \
+mkinst3.test \
+mmode.test \
+mmodely.test \
+multlib.test \
+no-extra-makefile-code.test \
+no-outdir-option.test \
+nobase.test \
+nobase-libtool.test \
+nobase-python.test \
+nodef.test \
+nodef2.test \
+nodep.test \
+nodep2.test \
+nodepcomp.test \
+nodist.test \
+nodist2.test \
+nodist3.test \
+nogzip.test \
+nogzip2.test \
+noinst.test \
+noinstdir.test \
+nolink.test \
+nostdinc.test \
+notrans.test \
+number.test \
+objc.test \
+objc2.test \
+obsolete.test \
+oldvars.test \
+order.test \
+output.test \
+output2.test \
+output3.test \
+output4.test \
+output5.test \
+output6.test \
+output7.test \
+output8.test \
+output9.test \
+output10.test \
+output11.test \
+output12.test \
+output13.test \
+output-order.test \
+override-conditional-1.test \
+override-conditional-2.test \
+override-html.test \
+override-suggest-local.test \
+parallel-am.test \
+parallel-am2.test \
+parallel-am3.test \
+parallel-tests.test \
+parallel-tests2.test \
+parallel-tests3.test \
+parallel-tests5.test \
+parallel-tests6.test \
+parallel-tests8.test \
+parallel-tests9.test \
+parallel-tests10.test \
+parallel-tests-suffix.test \
+parallel-tests-suffix-prog.test \
+parallel-tests-log-compiler-1.test \
+parallel-tests-log-compiler-2.test \
+parallel-tests-dry-run.test \
+parallel-tests-fd-redirect.test \
+parallel-tests-extra-programs.test \
+parallel-tests-unreadable.test \
+parallel-tests-subdir.test \
+parallel-tests-interrupt.test \
+parallel-tests-reset-term.test \
+parallel-tests-harderror.test \
+parallel-tests-log-override-1.test \
+parallel-tests-log-override-2.test \
+parallel-tests-log-override-recheck.test \
+parallel-tests-log-compiler-example.test \
+parallel-tests-cmdline-override.test \
+parallel-tests-fork-bomb.test \
+parallel-tests-empty-testlogs.test \
+parallel-tests-driver-install.test \
+parallel-tests-no-color-in-log.test \
+parallel-tests-no-spurious-summary.test \
+parallel-tests-exit-statuses.test \
+parallel-tests-console-output.test \
+parallel-tests-once.test \
+tests-environment.test \
+am-tests-environment.test \
+tests-environment-backcompat.test \
+testsuite-summary-color.test \
+testsuite-summary-count.test \
+testsuite-summary-count-many.test \
+testsuite-summary-reference-log.test \
+test-driver-custom-no-extra-driver.test \
+test-driver-custom.test \
+test-driver-custom-xfail-tests.test \
+test-driver-custom-multitest.test \
+test-driver-custom-multitest-recheck.test \
+test-driver-custom-multitest-recheck2.test \
+test-driver-custom-html.test \
+test-driver-custom-no-html.test \
+test-driver-create-log-dir.test \
+test-driver-strip-vpath.test \
+test-driver-trs-suffix-registered.test \
+test-driver-fail.test \
+test-harness-vpath-rewrite.test \
+test-log.test \
+test-logs-repeated.test \
+test-metadata-global-log.test \
+test-metadata-global-result.test \
+test-metadata-recheck.test \
+test-metadata-results.test \
+test-missing.test \
+test-missing2.test \
+test-trs-basic.test \
+test-trs-recover.test \
+test-trs-recover2.test \
+parse.test \
+percent.test \
+percent2.test \
+phony.test \
+pluseq.test \
+pluseq2.test \
+pluseq3.test \
+pluseq4.test \
+pluseq5.test \
+pluseq6.test \
+pluseq7.test \
+pluseq8.test \
+pluseq9.test \
+pluseq10.test \
+pluseq11.test \
+posixsubst-data.test \
+posixsubst-extradist.test \
+posixsubst-ldadd.test \
+posixsubst-libraries.test \
+posixsubst-ltlibraries.test \
+posixsubst-programs.test \
+posixsubst-scripts.test \
+posixsubst-sources.test \
+posixsubst-tests.test \
+postproc.test \
+ppf77.test \
+pr2.test \
+pr9.test \
+pr72.test \
+pr87.test \
+pr204.test \
+pr211.test \
+pr220.test \
+pr224.test \
+pr229.test \
+pr243.test \
+pr266.test \
+pr279.test \
+pr279-2.test \
+pr287.test \
+pr300-lib.test \
+pr300-ltlib.test \
+pr300-prog.test \
+pr307.test \
+pr401.test \
+pr401b.test \
+pr401c.test \
+prefix.test \
+primary.test \
+primary2.test \
+primary3.test \
+primary-prefix-invalid-couples.test \
+primary-prefix-valid-couples.test \
+primary-prefix-couples-force-valid.test \
+primary-prefix-couples-documented-valid.test \
+proginst.test \
+programs-primary-rewritten.test \
+python.test \
+python2.test \
+python3.test \
+python4.test \
+python5.test \
+python5b.test \
+python6.test \
+python7.test \
+python8.test \
+python9.test \
+python10.test \
+python11.test \
+python12.test \
+python-dist.test \
+python-vars.test \
+python-virtualenv.test \
+recurs.test \
+recurs2.test \
+remake.test \
+remake1a.test \
+remake2.test \
+remake3.test \
+remake3a.test \
+remake4.test \
+remake5.test \
+remake6.test \
+remake7.test \
+remake8a.test \
+remake8b.test \
+remake9a.test \
+remake9b.test \
+remake9c.test \
+remake9d.test \
+remake10a.test \
+remake10b.test \
+remake10c.test \
+remake11.test \
+remake12.test \
+remake-all-1.test \
+remake-all-2.test \
+remake-subdir-from-subdir.test \
+remake-subdir-gnu.test \
+remake-subdir.test \
+remake-subdir2.test \
+remake-subdir-long-time.test \
+remake-gnulib-add-acsubst.test \
+remake-gnulib-add-header.test \
+remake-gnulib-remove-header.test \
+remake-moved-m4-file.test \
+remake-deleted-m4-file.test \
+remake-renamed-m4-file.test \
+remake-renamed-m4-macro-and-file.test \
+remake-renamed-m4-macro.test \
+pr8365-remake-timing.test \
+regex.test \
+regex-obsolete.test \
+req.test \
+reqd.test \
+reqd2.test \
+repeated-options.test \
+rulepat.test \
+self-check-cleanup.test \
+self-check-dir.test \
+self-check-env-sanitize.tap \
+self-check-exit.tap \
+self-check-explicit-skips.test \
+self-check-is_newest.test \
+self-check-me.tap \
+self-check-reexec.test \
+self-check-report.test \
+self-check-sanity.test \
+self-check-seq.test \
+self-check-tap.test \
+self-check-unindent.test \
+sanity.test \
+scripts.test \
+seenc.test \
+silent.test \
+silent2.test \
+silent3.test \
+silent4.test \
+silent6.test \
+silent7.test \
+silent8.test \
+silent9.test \
+silentcxx.test \
+silentcxx-gcc.test \
+silentf77.test \
+silentf90.test \
+silent-many-gcc.test \
+silent-many-generic.test \
+silent-lex-gcc.test \
+silent-lex-generic.test \
+silent-yacc-gcc.test \
+silent-yacc-generic.test \
+silent-nowarn.test \
+silent-configsite.test \
+srcsub.test \
+srcsub2.test \
+space.test \
+specflg.test \
+specflg2.test \
+specflg3.test \
+specflg6.test \
+specflg7.test \
+specflg8.test \
+specflg9.test \
+specflg10.test \
+specflg-dummy.test \
+spell.test \
+spell2.test \
+spell3.test \
+spelling.test \
+spy.test \
+stdinc.test \
+stamph2.test \
+stdlib.test \
+stdlib2.test \
+strictness-override.test \
+strictness-precedence.test \
+strip.test \
+strip2.test \
+strip3.test \
+subdir.test \
+subdir2.test \
+subdir3.test \
+subdir4.test \
+subdir5.test \
+subdir6.test \
+subdir7.test \
+subdir8.test \
+subdir9.test \
+subdir10.test \
+subdirbuiltsources.test \
+subcond.test \
+subcond2.test \
+subcond3.test \
+subobj.test \
+subobj2.test \
+subobj4.test \
+subobj5.test \
+subobj6.test \
+subobj7.test \
+subobj8.test \
+subobj9.test \
+subobj10.test \
+subobj11a.test \
+subobj11b.test \
+subobj11c.test \
+subobjname.test \
+subpkg.test \
+subpkg2.test \
+subpkg3.test \
+subpkg4.test \
+subst.test \
+subst3.test \
+subst4.test \
+subst5.test \
+subst-no-trailing-empty-line.test \
+substref.test \
+substre2.test \
+substtarg.test \
+suffix.test \
+suffix2.test \
+suffix3.tap \
+suffix4.test \
+suffix5.test \
+suffix6.test \
+suffix6b.test \
+suffix6c.test \
+suffix7.test \
+suffix8.tap \
+suffix9.test \
+suffix10.tap \
+suffix11.tap \
+suffix12.test \
+suffix13.test \
+suffix-chain.tap \
+symlink.test \
+symlink2.test \
+syntax.test \
+$(tap_with_common_setup_TESTS) \
+$(tap_other_TESTS) \
+tags.test \
+tags2.test \
+tagsub.test \
+tar.test \
+tar2.test \
+tar3.test \
+target-cflags.test \
+targetclash.test \
+tests-environment-fd-redirect.test \
+tests-environment-and-log-compiler.test \
+txinfo.test \
+txinfo2.test \
+txinfo3.test \
+txinfo4.test \
+txinfo5.test \
+txinfo5b.test \
+txinfo6.test \
+txinfo7.test \
+txinfo8.test \
+txinfo9.test \
+txinfo10.test \
+txinfo13.test \
+txinfo16.test \
+txinfo17.test \
+txinfo18.test \
+txinfo19.test \
+txinfo20.test \
+txinfo21.test \
+txinfo22.test \
+txinfo23.test \
+txinfo24.test \
+txinfo25.test \
+txinfo26.test \
+txinfo27.test \
+txinfo28.test \
+txinfo29.test \
+txinfo30.test \
+txinfo31.test \
+txinfo32.test \
+txinfo33.test \
+txinfo-unrecognized-extension.test \
+transform.test \
+transform2.test \
+unused.test \
+upc.test \
+upc2.test \
+upc3.test \
+vala.test \
+vala1.test \
+vala2.test \
+vala3.test \
+vala4.test \
+vala5.test \
+vars.test \
+vars3.test \
+vartar.test \
+vartypos.test \
+vartypo2.test \
+version.test \
+version2.test \
+version3.test \
+version4.test \
+version6.test \
+version7.test \
+version8.test \
+vpath.test \
+vtexi.test \
+vtexi2.test \
+vtexi3.test \
+vtexi4.test \
+warnings-override.test \
+warnings-precedence.test \
+warnings-strictness-interactions.test \
+warnings-unknown.test \
+warnopts.test \
+warnings-win-over-strictness.test \
+warning-groups-win-over-strictness.test \
+werror.test \
+werror2.test \
+werror3.test \
+werror4.test \
+whoami.test \
+xsource.test \
+xz.test \
+yacc-basic.test \
+yacc-d-basic.test \
+yacc-clean.test \
+yacc.test \
+yacc2.test \
+yacc4.test \
+yacc5.test \
+yacc6.test \
+yacc7.test \
+yacc8.test \
+yaccdry.test \
+yacc-dist-nobuild.test \
+yacc-nodist.test \
+yaccpp.test \
+yaccvpath.test \
+yacc-d-vpath.test \
+yacc-dist-nobuild-subdir.test \
+yflags.test \
+yflags2.test \
+yflags-cmdline-override.test \
+yflags-conditional.test \
+yflags-d-false-positives.test \
+yflags-force-override.test \
+yflags-force-conditional.test \
+yflags-var-expand.test
+
+# List of tests on TAP support that use the files pre-computed by
+# `tap-common-setup.test', and source the `tap-setup.sh' helper
+# script.
+tap_with_common_setup_TESTS = \
+tap-ambiguous-directive.test \
+tap-autonumber.test \
+tap-bailout.test \
+tap-bailout-and-logging.test \
+tap-bailout-suppress-badexit.test \
+tap-bailout-suppress-later-diagnostic.test \
+tap-bailout-suppress-later-errors.test \
+tap-color.test \
+tap-deps.test \
+tap-diagnostic.test \
+tap-empty-diagnostic.test \
+tap-empty.test \
+tap-escape-directive.test \
+tap-escape-directive-2.test \
+tap-exit.test \
+tap-signal.test \
+tap-fancy.test \
+tap-fancy2.test \
+tap-global-log.test \
+tap-global-result.test \
+tap-html.test \
+tap-log.test \
+tap-msg0-result.test \
+tap-msg0-directive.test \
+tap-msg0-planskip.test \
+tap-msg0-bailout.test \
+tap-msg0-misc.test \
+tap-merge-stdout-stderr.test \
+tap-no-merge-stdout-stderr.test \
+tap-no-disable-hard-error.test \
+tap-no-spurious-summary.test \
+tap-no-spurious-numbers.test \
+tap-no-spurious.test \
+tap-not-ok-skip.test \
+tap-number-wordboundary.test \
+tap-numeric-description.test \
+tap-negative-numbers.test \
+tap-numbers-leading-zero.test \
+tap-out-of-order.test \
+tap-passthrough.test \
+tap-passthrough-exit.test \
+tap-plan.test \
+tap-plan-corner.test \
+tap-plan-errors.test \
+tap-plan-middle.test \
+tap-plan-whitespace.test \
+tap-plan-leading-zero.test \
+tap-plan-malformed.test \
+tap-missing-plan-and-bad-exit.test \
+tap-planskip.test \
+tap-planskip-late.test \
+tap-planskip-and-logging.test \
+tap-planskip-unplanned.test \
+tap-planskip-unplanned-corner.test \
+tap-planskip-case-insensitive.test \
+tap-planskip-whitespace.test \
+tap-planskip-badexit.test \
+tap-planskip-bailout.test \
+tap-planskip-later-errors.test \
+tap-realtime.test \
+tap-test-number-0.test \
+tap-recheck-logs.test \
+tap-result-comment.test \
+tap-todo-skip-together.test \
+tap-todo-skip-whitespace.test \
+tap-todo-skip.test \
+tap-unplanned.test \
+tap-whitespace-normalization.test \
+tap-with-and-without-number.test \
+tap-xfail-tests.test
+
+# Other tests on TAP support.
+tap_other_TESTS = \
+tap-common-setup.test \
+tap-bad-prog.tap \
+tap-basic.test \
+tap-diagnostic-custom.test \
+tap-driver-stderr.test \
+tap-doc.test \
+tap-doc2.test \
+tap-more.test \
+tap-more2.test \
+tap-recheck.test \
+tap-summary.test \
+tap-summary-color.test
+
+print-list-of-tests:
+ @echo $(handwritten_TESTS)
+.PHONY: print-list-of-tests
# Test for conditional libtool libraries.
# This combines two examples from the manual.
-required='libtoolize gcc'
+required='cc libtoolize'
. ./defs || Exit 1
cat >>configure.in <<'END'
test ! -f lib1bar.la
test ! -f lib2foo.la
test ! -f lib3foo.la
+
+:
# Test for bug in conditionals.
-required='libtoolize gcc'
+required='cc native libtoolize'
. ./defs || Exit 1
cat >> configure.in << 'END'
AC_PROG_CC
AC_PROG_LIBTOOL
AC_SUBST([HELLO_SYSTEM], [hello-generic.lo])
-AM_CONDITIONAL([LINUX], true)
+AM_CONDITIONAL([LINUX], [true])
AC_OUTPUT
END
: > check-ok
END
-echo 'char* str() { return "hello-linux"; }' >hello-linux.c
-echo 'char* str() { return "hello-generic"; }' >hello-generic.c
-cat >hello-common.c <<'END'
+cat > hello-linux.c <<'END'
+const char* str (void)
+{
+ return "hello-linux";
+}
+END
+
+cat > hello-generic.c <<'END'
+const char* str (void)
+{
+ return "hello-generic";
+}
+END
+
+cat > hello-common.c <<'END'
#include <stdio.h>
-char* str();
-void print()
+const char* str (void);
+void print (void)
{
puts (str ());
}
END
-echo 'int main() { print(); return 0; }' >main.c
+
+cat > main.c <<'END'
+int main (void)
+{
+ print();
+ return 0;
+}
+END
libtoolize
$ACLOCAL
./configure
$MAKE check
test -f check-ok
+
+:
# Test for libtool convenience libraries.
# This example is taken from the manual.
-required='libtoolize gcc'
+required='cc native libtoolize'
. ./defs || Exit 1
cat >>configure.in <<'END'
AC_PROG_CC
AC_PROG_LIBTOOL
-AC_CONFIG_FILES(sub1/Makefile
- sub2/Makefile
- sub2/sub21/Makefile
- sub2/sub22/Makefile)
+AC_CONFIG_FILES([sub1/Makefile
+ sub2/Makefile
+ sub2/sub21/Makefile
+ sub2/sub22/Makefile])
AC_OUTPUT
END
echo 'int sub22 () { return 22; }' > sub2/sub22/sub22.c
-cat >test.c <<EOF
+cat >test.c <<'EOF'
#include <stdio.h>
int main ()
{
test 0 = `wc -l < lib.lst`
find bin -type f -print > bin.lst
test 0 = `wc -l < bin.lst`
+
+:
# interface (with LT_INIT etc.) is used (this interface has been
# present since libtool 1.9b, circa 2004).
-required='libtoolize'
+required='cc libtoolize'
. ./defs || Exit 1
cat >>configure.in <<'END'
libtoolize
# Skip if older libtool (pre-2.0) is used.
{ $ACLOCAL && $AUTOCONF; } || {
- if test $? -eq 63; then Exit 77; else Exit 1; fi
+ if test $? -eq 63; then
+ skip_ "libtool is too old (probably < 2.0)"
+ else
+ Exit 1
+ fi
}
$EGREP 'LT_(INIT|PREREQ)' configure && Exit 1 # Sanity check.
$AUTOMAKE -a
cwd=`pwd`
./configure --prefix="$cwd/inst" >stdout || { cat stdout; Exit 1; }
cat stdout
-grep '^checking.*dlopen' stdout
+grep '^checking.*dlfcn\.h.* no$' stdout || grep '^checking.*dlopen' stdout
$MAKE
$MAKE install
# Test to make sure a sensible default source for libraries is used.
-required='libtool gcc'
+required='cc libtool'
. ./defs || Exit 1
cat >> configure.in << 'END'
END
cat > foo.c << 'END'
-int foo (void) { return 0; }
+int foo (void)
+{
+ return 0;
+}
END
cp foo.c zoo.d/bar.c
$ACLOCAL
$AUTOCONF
AUTOMAKE_fails -a
-grep 'Makefile.am:2:.*old_la.c' stderr
-grep 'Makefile.am:2:.*old.c' stderr
-grep 'Makefile.am:3:.*zoo_d_old2_la.c' stderr
-grep 'Makefile.am:3:.*zoo\.d/old2.c' stderr
+grep '^Makefile\.am:2:.*old_la\.c' stderr
+grep '^Makefile\.am:2:.*old\.c' stderr
+grep '^Makefile\.am:3:.*zoo_d_old2_la\.c' stderr
+grep '^Makefile\.am:3:.*zoo\.d/old2\.c' stderr
$AUTOMAKE -Wno-obsolete
test -f zoo.d/old2.la
$MAKE distcheck
+
+:
# Test for correct installation order of nobase libtool libraries.
-required='libtoolize'
+required='cc libtoolize'
. ./defs || Exit 1
cat >>configure.in <<'END'
$MAKE uninstall
test `find inst -type f -print | wc -l` -eq 0
+
:
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Check support for no-dist-gzip with lzip
+# Check support for no-dist-gzip with lzip.
required=lzip
. ./defs || Exit 1
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Check support for no-dist-gzip with lzma
+# Check support for no-dist-gzip with lzma.
required=lzma
. ./defs || Exit 1
cd build
../configure "--prefix=`pwd`/inst"
-$MAKE -j2 || Exit 77
+$MAKE -j2 || skip_ "$MAKE failed to run with two parallel jobs"
$MAKE -j2 distcheck
$MAKE test-distdir-removed
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Ensure that `make -n dist' and `make -n distcheck' show what would happen.
-# This currently works for GNU make only.
+# Ensure that `make -n dist' and `make -n distcheck' show what would
+# happen. This currently works for GNU make, but might fail with other
+# make implementations (e.g., FreeBSD's or NetBSD's).
# Please keep this test in sync with sister test maken.test.
done
}
-$AUTOMAKE -Wno-override
+$AUTOMAKE -a -Wno-override
./configure
check_targets || Exit 1
# Please keep this test in sync with sister test maken3.test.
-required=GNUmake
. ./defs || Exit 1
+if using_gmake; then
+ : Good, make is GNU make.
+else
+ unindent > mk.tmp << 'END'
+ targ.tmp:
+ : > $@
+ .MAKE: targ.tmp
+END
+ if $MAKE -n -f mk.tmp targ.tmp && test -f targ.tmp; then
+ : Good, make supports the '.MAKE' special target.
+ else
+ skip_ "GNU or BSD make required"
+ fi
+ rm -f *.tmp
+fi
+
mkdir sub sub2
cat >> configure.in << 'END'
done
}
-$AUTOMAKE -Wno-override
+$AUTOMAKE -a -Wno-override
./configure
check_targets || Exit 1
mkdir bin
cat > bin/help2man <<'END'
#! /bin/sh
-# fake help2man script that lets `missing' think it is not installed
+# Fake help2man script that lets `missing' think it is not installed.
exit 127
END
chmod +x bin/help2man
cd build
../configure
-# sanity check
+# Sanity check.
grep '^HELP2MAN *=.*/missing --run help2man' Makefile
$MAKE
$MAKE distcheck
cd ..
-rm -f *.1 # remove leftover generated manpages
+rm -f *.1 # Remove leftover generated manpages.
./configure
-# sanity check
+# Sanity check.
grep '^HELP2MAN *=.*/missing --run help2man' Makefile
$MAKE
cat > Makefile.am << 'END'
dist_man_MANS = foo.1
foo.1:
-## this 'rm' command will fail if $(srcdir) is unwritable
+## This 'rm' command will fail if $(srcdir) is unwritable.
rm -f $(srcdir)/$@
: > $(srcdir)/$@
END
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Test to make sure that mdate-sh is added to the right directory.
-# Report from Kevin Dalley
+# Report from Kevin Dalley.
. ./defs || Exit 1
# Test to make sure mdate-sh works correctly.
. ./defs || Exit 1
-cat >> configure.in << 'END'
-AC_OUTPUT
-END
-cat > Makefile.am << 'END'
-info_TEXINFOS = textutils.texi
-END
+get_shell_script mdate-sh
-cat > textutils.texi << 'END'
-@include version.texi
-@setfilename textutils.info
-END
-
-$ACLOCAL
-$AUTOMAKE --add-missing
-
-set x `$SHELL ./mdate-sh Makefile.am`
+set x `./mdate-sh install-sh`
shift
+echo "$*" # For debugging.
+
# Check that mdate output looks like a date:
test $# = 3
case $1$3 in *[!0-9]*) Exit 1;; esac
test $1 -lt 32
+# Hopefully automake will be obsolete in 80 years ;-)
+case $3 in 20[0-9][0-9]) :;; *) Exit 1;; esac
+case $2 in
+ January|February|March|April|May|June|July|August) ;;
+ September|October|November|December) ;;
+ *) Exit 1
+esac
+
+# Stricter checks on the year required a POSIX date(1) command.
+if year=`date +%Y` && test $year -gt 2010; then
+ test $year = $3 || Exit 1
+fi
+
+:
a=
file='file name $a'
-( : > "$file" ) || Exit 77
+( : > "$file" ) \
+ || skip_ "file name with spaces and metacharacters not accepted"
-cp "$top_testsrcdir/lib/mdate-sh" .
-$SHELL ./mdate-sh "$file" >stdout 2>stderr ||
+get_shell_script mdate-sh
+
+./mdate-sh "$file" >stdout 2>stderr ||
{ cat stdout; cat stderr >&2; Exit 1; }
cat stdout
cat stderr >&2
: > Makefile.am
+get_shell_script missing
+
$ACLOCAL
$AUTOCONF
$AUTOMAKE --add-missing
: > Makefile.am
+get_shell_script missing
+
$ACLOCAL
$AUTOCONF
$AUTOMAKE --add-missing
. ./defs || Exit 1
+get_shell_script missing
+
# b7cb8259 assumed not to exist.
./missing b7cb8259 --version 2>stderr && { cat stderr >&2; Exit 1; }
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# test how well `missing' finds output file names of various tools.
+# Test how well `missing' finds output file names of various tools.
# PR automake/483.
. ./defs || Exit 1
-# these programs may be invoked by `missing'
+get_shell_script missing
+
+# These programs may be invoked by `missing'.
needed_tools='chmod find sed test touch'
needed_tools_csep=`echo $needed_tools | sed 's/ /, /g'`
EOF
for tool in $needed_tools; do
- cat >$tool.in <<EOF
-#! /bin/sh
-exec @$tool@ "\$@"
+ unindent >$tool.in <<EOF
+ #! /bin/sh
+ exec @$tool@ "\$@"
EOF
done
$MAKE
sed 's/^dnl!! //' < configure.ac > configure.tmp
-cmp configure.ac configure.tmp && Exit 99 # sanity check
+cmp configure.ac configure.tmp && fatal_ 'failed to edit configure.ac'
mv -f configure.tmp configure.ac
$MAKE 2>stderr || { cat stderr >&2; Exit 1; }
. ./defs || Exit 1
# Make sure the directory we will create can be created...
-mkdir '~a b' || Exit 77
-mkdir '~a b/-x y' || Exit 77
+mkdir '~a b' && mkdir '~a b/-x y' \
+ || skip_ "directory names with spaces and metacharacters not accepted"
rm -rf '~a b'
-cp "$top_testsrcdir/lib/mkinstalldirs" .
+get_shell_script mkinstalldirs
# Test mkinstalldirs with the installed mkdir.
# sources in maintainer mode.
# From Derek R. Price.
-required=gcc
+required=cc
. ./defs || Exit 1
cat >> configure.in << 'END'
LDADD = @LEXLIB@
END
-# The point of this test is that it is not dependent on a working lex or yacc.
+# The point of this test is that it is not dependent on a working lex
+# or yacc.
cat > joe.c <<EOF
int joe (int arg)
{
chmod +x myyacc.sh mylex.sh
PATH="`pwd`$PATH_SEPARATOR$PATH"
-# make maintainer-clean; ./configure; make should always work,
+# "make maintainer-clean; ./configure; make" should always work,
# per GNU Standard.
$MAKE maintainer-clean
./configure
# This is just the libtool equivalent of nobase.test, split
# up for greater exposure of nobase.test.
-required='libtoolize gcc'
+required='cc libtoolize'
. ./defs || Exit 1
cat >> configure.in <<'EOF'
test -f inst/foo/sub/libnobase.la
test ! -f inst/foo/libnobase.la
test -f inst/foo/libbase.la
+
+.PHONY: test-install-exec test-install-data
EOF
mkdir sub
$MAKE test-install-exec
$MAKE uninstall
test `find inst/foo -type f -print | wc -l` = 0
+
+:
nobase_my_PYTHON = two.py sub/nobase.py
test-install-data: install-data
- find inst -print; : for debugging
+ find inst -print; : For debugging.
test -f inst/my/one.py
test -f inst/my/one.pyc
test -f inst/my/two.py
# Make sure nobase_* works.
-required='gcc'
+required=cc
. ./defs || Exit 1
cat >> configure.in <<'EOF'
test -f inst/foo/sub/libnobase.a
test ! -f inst/foo/libnobase.a
test -f inst/foo/libbase.a
+
+.PHONY: test-install-exec test-install-data
EOF
mkdir sub
$MAKE test-install-exec
$MAKE uninstall
test `find inst/foo -type f -print | wc -l` = 0
+
+:
# Test to make sure nostdinc option works correctly.
+# We don't require a C compiler explicitly, because the first part of the
+# test (where `Makefile.in' is grepped) does not require one. Insted, we
+# just skip the rest of the test if configure fails to find a working C
+# compiler.
+
. ./defs || Exit 1
cat >> configure.in << 'END'
$ACLOCAL
$AUTOMAKE
-grep '.*-I *\.' Makefile.in && Exit 1
+$EGREP '(-I *\.|-I.*srcdir|am__isrc)' Makefile.in && Exit 1
# We'll test the fully-processed Makefile too.
$AUTOCONF
# Test with $builddir != $srcdir
mkdir build
cd build
-../configure
+../configure || Exit $?
$EGREP '.*-I *(\.|\$.srcdir.)' Makefile && Exit 1
# Test with $builddir = $srcdir
cd ..
-./configure
+./configure || Exit $?
$EGREP '.*-I *(\.|\$.srcdir.)' Makefile && Exit 1
Exit 0
notrans_dist_man2_MANS = z-bar.man z-bar2.man
notrans_nodist_man1_MANS = z-baz.man z-baz2.man
-# These two are ignored
+# These two are ignored.
dist_notrans_man_MANS = nosuch.8
nodist_notrans_man9_MANS = nosuch.man
cat obs >> configure.in
$PERL -ne 'chomp; print "grep $_ output || Exit 1\n"; ' obs > obs.1
$PERL -ne 'chomp; print "grep $_ configure.in && Exit 1\n"; ' obs > obs.2
-echo : >> obs.1 # since it will be sourced, it must and with a success
-echo : >> obs.2 # ditto
+echo : >> obs.1 # Since it will be sourced, it must end with a success.
+echo : >> obs.2 # Likewise.
-cat configure.in # for debugging
-cat obs.1 # ditto
-cat obs.2 # ditto
+cat configure.in # For debugging.
+cat obs.1 # Likewise.
+cat obs.2 # Likewise.
# Sanity check. Make sure we have added something to configure.in.
test `wc -l <configure.in` -gt 1
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Check for support for colon separated input files in AC_CONFIG_FILES
+# Check for support for colon-separated input files in AC_CONFIG_FILES.
-required=GNUmake
. ./defs || Exit 1
cat > configure.in << END
AC_INIT([$me], [1.0])
AM_INIT_AUTOMAKE
-AC_CONFIG_FILES(foo:a.in:b.in:c.in:d.in)
+AC_CONFIG_FILES([foo:a.in:b.in:c.in:d.in])
AC_OUTPUT
END
c = C
d = C
-print:
- @echo $a$b$c$d
+.PHONY: test1 test2
+test1:
+ test $(a)$(b)$(c)$(d) = ABCD
+test2:
+ test $(a)$(b)$(c)$(d) = GFCD
EOF
echo 'd = D' > d.in
$AUTOCONF
$AUTOMAKE
./configure
-$MAKE -f foo print | grep 'ABCD'
+$MAKE -f foo test1
$sleep
cat >b.in <<'EOF'
d = F
EOF
-# This involves the rebuild rule, hence it requires GNUmake
-$MAKE -f foo print | grep 'GFCD'
+using_gmake || $MAKE -f foo
+$MAKE -f foo test2
+
+:
-
#! /bin/sh
# Copyright (C) 2010, 2011 Free Software Foundation, Inc.
#
$ACLOCAL
$AUTOMAKE -Wno-override
AUTOMAKE_fails
-grep ':.*install-local' stderr && Exit 1 # There is no such thing as install-local
+# There is no such thing as `install-local'.
+grep ':.*install-local' stderr && Exit 1
grep ':.*installcheck-local' stderr
grep ':.*html-local' stderr
# Make sure overriding *-am targets suggest using *-local, not *-am.
# Generate expected output using the non-threaded code.
unset AUTOMAKE_JOBS || :
-AUTOMAKE_run 0 --add-missing
+AUTOMAKE_run --add-missing
mv stderr expected
Makefile_ins=`find . -name Makefile.in`
for file in $Makefile_ins; do
for run in 1 2 3 4 5 6 7; do
rm -f build-aux/* sub*/Makefile.in
- AUTOMAKE_run 0 --add-missing
+ AUTOMAKE_run --add-missing
diff stderr expected
for file in $Makefile_ins; do
diff $file $file.exp
# This tests:
# 4) warning and normal error output should be identical, in that duplicate
-# warnings should be omitted in the same way as without threads,
+# warnings should be omitted in the same way as without threads.
required=perl-threads
. ./defs || Exit 1
echo "AC_CONFIG_FILES([sub$i/Makefile])" >> configure.in
echo "SUBDIRS += sub$i" >> Makefile.am
mkdir sub$i sub$i/sub
- cat > sub$i/Makefile.am << END
-AUTOMAKE_OPTIONS = subdir-objects
-bin_PROGRAMS = sub$i
-sub${i}_SOURCES = sub/main$i.c
+ unindent > sub$i/Makefile.am << END
+ AUTOMAKE_OPTIONS = subdir-objects
+ bin_PROGRAMS = sub$i
+ sub${i}_SOURCES = sub/main$i.c
END
done
# This tests:
# 3) normal automake output should be identical and ordered in the same way
# with --add-missing, even with concurrent file requirements, and the
-# installation of aux files should be race-free,
+# installation of aux files should be race-free.
required=perl-threads
. ./defs || Exit 1
echo "AC_CONFIG_FILES([sub$i/Makefile])" >> configure.in
echo "SUBDIRS += sub$i" >> Makefile.am
mkdir sub$i
- cat > sub$i/Makefile.am <<END
-python_PYTHON = foo$i.py
-lisp_LISP = foo$i.el
-bin_PROGRAMS = p$i
+ unindent > sub$i/Makefile.am <<END
+ python_PYTHON = foo$i.py
+ lisp_LISP = foo$i.el
+ bin_PROGRAMS = p$i
END
done
# Generate expected output using the non-threaded code.
unset AUTOMAKE_JOBS || :
-AUTOMAKE_run 0 --add-missing
+AUTOMAKE_run --add-missing
mv stderr expected
mv Makefile.in Makefile.in.exp
for run in 1 2 3 4 5 6 7; do
rm -f build-aux/* sub*/Makefile.in
- AUTOMAKE_run 0 --add-missing
+ AUTOMAKE_run --add-missing
diff stderr expected
diff Makefile.in Makefile.in.exp
done
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# parallel-tests: some checks on console output about testsuite
+# progress.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+XFAIL_TESTS = sub/xpass.test xfail.test error.test
+TESTS = $(XFAIL_TESTS) fail.test pass.test a/b/skip.test sub/error2.test
+pass.log: fail.log
+error.log: pass.log
+sub/xpass.log: error.log
+sub/error2.log: xfail.log
+a/b/skip.log: sub/error2.log
+END
+
+cat > exp <<'END'
+FAIL: fail.test
+PASS: pass.test
+ERROR: error.test
+XPASS: sub/xpass.test
+XFAIL: xfail.test
+ERROR: sub/error2.test
+SKIP: a/b/skip.test
+END
+
+mkdir sub a a/b
+
+cat > pass.test << 'END'
+#!/bin/sh
+exit 0
+END
+cp pass.test sub/xpass.test
+
+cat > fail.test << 'END'
+#!/bin/sh
+exit 1
+END
+
+cat > xfail.test << 'END'
+#!/bin/sh
+# The sleep should ensure expected execution order of tests
+# even when make is run in parallel mode.
+# FIXME: quotes below required by maintainer-check.
+sleep '10'
+exit 1
+END
+
+cat > error.test << 'END'
+#!/bin/sh
+exit 99
+END
+cp error.test sub/error2.test
+
+cat > a/b/skip.test << 'END'
+#!/bin/sh
+exit 77
+END
+
+chmod a+x pass.test fail.test xfail.test sub/xpass.test \
+ a/b/skip.test error.test sub/error2.test
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+for vpath in : false; do
+ if $vpath; then
+ mkdir build
+ cd build
+ srcdir=..
+ else
+ srcdir=.
+ fi
+ $srcdir/configure
+ $MAKE check >stdout && { cat stdout; Exit 1; }
+ cat stdout
+ LC_ALL=C grep '^[A-Z][A-Z]*:' stdout > got
+ cat got
+ diff $srcdir/exp got
+ cd $srcdir
+done
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check that auxiliary script 'test-driver' gets automatically installed
+# in the correct directory by 'parallel-tests' option.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+: Try first with parallel-tests defined in AM_INIT_AUTOMAKE.
+
+mkdir am-init-automake
+cd am-init-automake
+
+cat > configure.in <<END
+AC_INIT([$me], [1.0])
+AC_CONFIG_AUX_DIR([my_aux_dir])
+AM_INIT_AUTOMAKE([parallel-tests])
+AC_CONFIG_FILES([Makefile sub/Makefile])
+AC_OUTPUT
+END
+
+mkdir sub my_aux_dir
+
+cat > Makefile.am <<END
+SUBDIRS = sub
+TESTS = foo.test
+END
+
+cat > sub/Makefile.am <<END
+TESTS = bar.test
+END
+
+$ACLOCAL
+$AUTOMAKE -a 2>stderr || { cat stderr >&2; Exit 1; }
+cat stderr >&2
+
+ls -l . sub my_aux_dir # For debugging.
+test -f my_aux_dir/test-driver
+test ! -r test-driver
+test ! -r sub/test-driver
+
+grep '^configure\.in:3:.*installing.*my_aux_dir/test-driver' stderr
+
+cd ..
+
+: Now try with parallel-tests defined in AUTOMAKE_OPTIONS.
+
+mkdir automake-options
+cd automake-options
+
+cat > configure.in <<END
+AC_INIT([$me], [1.0])
+AC_CONFIG_AUX_DIR([build-aux])
+AM_INIT_AUTOMAKE
+AC_CONFIG_FILES([dir/GNUmakefile])
+AC_OUTPUT
+END
+
+mkdir build-aux dir
+
+cat > dir/GNUmakefile.am <<END
+TESTS = foo.test
+AUTOMAKE_OPTIONS = parallel-tests
+TESTS += bar.test
+END
+
+$ACLOCAL
+$AUTOMAKE --add-missing --copy dir/GNUmakefile 2>stderr \
+ || { cat stderr >&2; Exit 1; }
+cat stderr >&2
+
+ls -l . dir build-aux # For debugging.
+test -f build-aux/test-driver
+test ! -r test-driver
+test ! -r dir/test-driver
+
+grep '^dir/GNUmakefile\.am:2:.*installing.*build-aux/test-driver' stderr
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check `make -n' for testsuite-related targets, when `parallel-tests'
+# is in use.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
+
+cat > Makefile.am <<'END'
+TESTS = foo.test bar.test
+TEST_LOG_COMPILER = $(SHELL)
+END
+
+$ACLOCAL
+$AUTOMAKE -a
+$AUTOCONF
+
+./configure
+
+make_n_ ()
+{
+ st=0
+ $MAKE -n "$@" >output 2>&1 || { cat output; ls -l; Exit 1; }
+ cat output
+ # Look out for possible errors from common tools used by recipes.
+ $EGREP -i ' (exist|permission|denied|no .*(such|file))' output && Exit 1
+ $EGREP '(mv|cp|rm|cat|grep|sed|awk): ' output && Exit 1
+ :
+}
+
+: > output
+
+files='foo.log bar.log foo.trs bar.trs'
+
+for target in check recheck test-suite.log; do
+ test ! -f foo.log
+ test ! -f foo.trs
+ test ! -f bar.log
+ test ! -f bar.trs
+ test ! -f test-suite.log
+done
+
+# FIXME: creative quoting below to please maintainer-check.
+echo exit '0' > foo.test
+echo exit '1' > bar.test
+
+$MAKE check && Exit 1
+
+chmod a-w .
+
+make_n_ clean
+test -f foo.log
+test -f foo.trs
+test -f foo.log
+test -f bar.trs
+
+# FIXME: creative quoting below to please maintainer-check.
+cat > foo.test <<END
+echo this is bad
+exit 1
+END
+echo exit '0' > bar.test
+
+for target in check recheck test-suite.log; do
+ make_n_ $target
+ grep '^:test-result: *FAIL' bar.trs
+ grep 'this is bad' foo.log test-suite.log && Exit 1
+done
+
+chmod a-rw $files
+if test -r foo.log; then
+ : You can still read unreadable files! Skip these checks.
+else
+ for target in check recheck test-suite.log; do
+ make_n_ $target
+ for f in $files; do
+ test -f $f && test ! -r $f || Exit 1
+ done
+ done
+fi
+chmod u+r $files
+
+chmod u+w .
+rm -f foo.log bar.trs
+chmod a-w .
+for target in check recheck test-suite.log $files; do
+ make_n_ $target
+ test ! -f foo.log
+ test -f foo.trs
+ test ! -f bar.trs
+ test -f bar.log
+done
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check parallel-tests features:
+# - empty TESTS
+# - empty TEST_LOGS
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.in << 'END'
+AC_CONFIG_FILES([sub1/Makefile sub2/Makefile])
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+SUBDIRS = sub1 sub2
+END
+
+mkdir sub1 sub2
+
+cat > sub1/Makefile.am << 'END'
+TESTS =
+check-local:
+ echo $(TEST_LOGS) | grep . && exit 1; exit 0
+END
+
+cat > sub2/Makefile.am << 'END'
+TESTS = foo.test
+END
+
+cat > sub2/foo.test <<'END'
+#! /bin/sh
+exit 0
+END
+chmod a+x sub2/foo.test
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+no_test_has_run ()
+{
+ ls -1 *.log | grep -v '^test-suite\.log$' | grep . && Exit 1
+ grep '^# TOTAL: *0$' test-suite.log
+ :
+}
+
+for vpath in : false; do
+ if $vpath; then
+ mkdir build
+ cd build
+ srcdir=..
+ else
+ srcdir=.
+ fi
+ $srcdir/configure
+ cd sub1
+ VERBOSE=yes $MAKE check
+ no_test_has_run
+ cd ../sub2
+ VERBOSE=yes TESTS='' $MAKE -e check
+ no_test_has_run
+ VERBOSE=yes TEST_LOGS='' $MAKE -e check
+ no_test_has_run
+ cd ..
+ $MAKE check
+ cat sub2/foo.log
+ $MAKE distclean
+ cd $srcdir
+done
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check parallel-tests features: normal and special exit statuses
+# in the test scripts.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
+
+# $failure_statuses should be defined to the list of all integers between
+# 1 and 255 (inclusive), excluded 77 and 99.
+failure_statuses=`seq_ 1 255 | $EGREP -v '^(77|99)$' | tr "$nl" ' '`
+# For debugging.
+echo "failure_statuses: $failure_statuses"
+# Sanity check.
+test `for st in $failure_statuses; do echo $st; done | wc -l` -eq 253 \
+ || fatal_ "initializing list of exit statuses for simple failures"
+
+cat > Makefile.am <<END
+LOG_COMPILER = ./do-exit
+fail_tests = $failure_statuses
+TESTS = 0 77 99 $failure_statuses
+\$(TESTS):
+END
+
+cat > do-exit <<'END'
+#!/bin/sh
+echo "$0: $1"
+case $1 in
+ [0-9]|[0-9][0-9]|[0-9][0-9][0-9]) st=$1;;
+ */[0-9]|*/[0-9][0-9]|*/[0-9][0-9][0-9]) st=`echo x"$1" | sed 's|.*/||'`;;
+ *) st=99;;
+esac
+exit $st
+END
+chmod a+x do-exit
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+{
+ echo PASS: 0
+ echo SKIP: 77
+ echo ERROR: 99
+ for st in $failure_statuses; do
+ echo "FAIL: $st"
+ done
+} | LC_ALL=C sort > exp-fail
+
+sed 's/^FAIL:/XFAIL:/' exp-fail | LC_ALL=C sort > exp-xfail-1
+sed '/^ERROR:/d' exp-xfail-1 > exp-xfail-2
+
+sort exp-fail
+sort exp-xfail-1
+sort exp-xfail-2
+
+./configure
+
+st=1
+$MAKE check >stdout && st=0
+cat stdout
+cat test-suite.log
+test $st -gt 0 || Exit 1
+LC_ALL=C grep '^[A-Z][A-Z]*:' stdout | LC_ALL=C sort > got-fail
+diff exp-fail got-fail
+
+st=1
+XFAIL_TESTS="$failure_statuses 99" $MAKE -e check >stdout && st=0
+cat stdout
+cat test-suite.log
+test $st -gt 0 || Exit 1
+LC_ALL=C grep '^[A-Z][A-Z]*:' stdout | LC_ALL=C sort > got-xfail-1
+diff exp-xfail-1 got-xfail-1
+
+st=0
+XFAIL_TESTS="$failure_statuses" TESTS="0 77 $failure_statuses" \
+ $MAKE -e check >stdout || st=$?
+cat stdout
+cat test-suite.log
+test $st -eq 0 || Exit 1
+LC_ALL=C grep '^[A-Z][A-Z]*:' stdout | LC_ALL=C sort > got-xfail-2
+diff exp-xfail-2 got-xfail-2
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Parallel test harness: check that $(TESTS) can lazily depend on
+# (or even be) $(EXTRA_PROGRAMS).
+
+required='cc native'
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.in << 'END'
+AC_PROG_CC
+AC_OUTPUT
+END
+
+# Will be extended later.
+cat > Makefile.am << 'END'
+TEST_EXTENSIONS = .bin .test
+EXTRA_PROGRAMS =
+TESTS =
+END
+
+#
+# Now try various kinds of test dependencies ...
+#
+
+# 1. A program that is also a test, and whose source files
+# already exist.
+
+cat >> Makefile.am <<'END'
+EXTRA_PROGRAMS += foo.bin
+TESTS += foo.bin
+foo_bin_SOURCES = foo.c
+END
+
+cat > foo.c <<'END'
+#include <stdio.h>
+int main (void)
+{
+ printf ("foofoofoo\n");
+ return 0;
+}
+END
+
+# 2. A program that is also a test, and whose source files
+# are buildable by make.
+cat >> Makefile.am <<'END'
+EXTRA_PROGRAMS += bar.bin
+TESTS += bar.bin
+bar_bin_SOURCES = bar.c
+bar.c: foo.c
+ sed -e 's/foofoofoo/barbarbar/' foo.c > $@
+END
+
+# 3. A test script that already exists, whose execution depends
+# on a program whose source files already exist and which is
+# not itself a test.
+cat >> Makefile.am <<'END'
+EXTRA_PROGRAMS += y
+TESTS += baz.test
+baz.log: y$(EXEEXT)
+END
+
+cat > baz.test <<'END'
+#!/bin/sh
+$srcdir/y "$@" | sed 's/.*/&ep&ep&ep/'
+END
+chmod a+x baz.test
+
+cat > y.c <<'END'
+#include <stdio.h>
+int main (void)
+{
+ printf ("y\n");
+ return 0;
+}
+END
+
+# 4. A program that is also a test, but whose source files
+# do not exit and are not buildable by make.
+
+cat >> Makefile.am <<'END'
+EXTRA_PROGRAMS += none.bin
+TESTS += none.bin
+none_bin_SOURCES = none.c
+END
+
+#
+# Setup done, go with the tests.
+#
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+./configure
+
+# What we check now:
+# 1. even if we cannot build the `none.bin' program, all the other
+# test programs should be built, and all the other tests should
+# be run;
+# 2. still, since we cannot create the `none.log' file, the
+# `test-suite.log' file shouldn't be created (as it depends
+# on *all* the test logs).
+
+st=0
+$MAKE check >stdout 2>stderr || st=$?
+cat stdout
+cat stderr >&2
+ls -l
+test $st -gt 0 || Exit 1
+
+# Files that should have been created, with the expected content.
+cat bar.c
+grep foofoofoo foo.log
+grep barbarbar bar.log
+grep yepyepyep baz.log
+# Files that shouldn't have been created.
+test ! -f none.log
+test ! -f test-suite.log
+# Expected testsuite progress output.
+grep '^PASS: baz\.test$' stdout
+# Don't anchor the end of the next two patterns, to allow for non-empty
+# $(EXEEXT).
+grep '^PASS: foo\.bin' stdout
+grep '^PASS: bar\.bin' stdout
+# Expected error messages from make.
+$EGREP 'none\.(bin|o|c)' stderr
+
+# What we check now:
+# 1. if we make the last EXTRA_PROGRAM buildable, the failed tests
+# pass;
+# 2. on a lazy re-run, the passed tests are not re-run, and
+# 3. their log files are not updated or touched.
+
+: > stamp
+$sleep
+
+echo 'int main (void) { return 0; }' > none.c
+
+st=0
+RECHECK_LOGS= $MAKE -e check >stdout || st=$?
+cat stdout
+ls -l
+test $st -eq 0 || Exit 1
+
+# For debugging.
+stat stamp foo.log bar.log baz.log || :
+
+# Files that shouldn't have been updated or otherwise touched.
+is_newest stamp foo.log bar.log baz.log
+# Files that should have been created now.
+test -f none.log
+test -f test-suite.log
+# Tests that shouldn't have been re-run.
+$EGREP '(foo|bar)\.bin|baz\.test$' stdout && Exit 1
+# Tests that should have been run. Again, we don't anchor the end
+# of the next pattern, to allow for non-empty $(EXEEXT).
+grep '^PASS: none\.bin' stdout
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# parallel-tests support: redirection of file descriptors with
+# AM_TESTS_FD_REDIRECT, even when using tests without suffix, or
+# which are binary executables.
+# We use some tricks to ensure that all code paths in `lib/am/check2.am'
+# are covered, even on platforms where $(EXEEXT) would be naturally empty.
+# See also the more generic test 'check-fd-redirect.test'.
+
+required='cc native'
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.in << 'END'
+AC_PROG_CC
+AM_CONDITIONAL([real_EXEEXT], [test -n "$EXEEXT"])
+test -n "$EXEEXT" || EXEEXT=.bin
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+AM_TESTS_FD_REDIRECT = 9>&1
+TEST_EXTENSIONS = .test .sh
+TESTS = foo.sh bar $(check_PROGRAMS)
+check_PROGRAMS = baz qux.test
+qux_test_SOURCES = zardoz.c
+
+## Sanity check.
+if !real_EXEEXT
+check-local:
+ test -f baz.bin
+ test -f qux.test.bin
+endif
+END
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+cat > foo.sh <<'END'
+#!/bin/sh
+echo " foofoofoo" >&9
+END
+chmod a+x foo.sh
+
+cat > bar <<'END'
+#!/bin/sh
+echo " barbarbar" >&9
+END
+
+chmod a+x foo.sh bar
+
+cat > baz.c <<'END'
+#include <unistd.h>
+int main (void)
+{
+ write (9, " bazbazbaz\n", 11);
+ return 0;
+}
+END
+
+cat > zardoz.c <<'END'
+#include <unistd.h>
+int main (void)
+{
+ write (9, " quxquxqux\n", 11);
+ return 0;
+}
+END
+
+./configure
+
+# Sanity checks.
+st=0
+grep '^bar\.log:.*bar' Makefile || st=1
+grep '^baz\.log:.*baz\$(EXEEXT)' Makefile || st=1
+grep '^\.test\$(EXEEXT)\.log:' Makefile || st=1
+$EGREP '^(foo|qux)\.log:' Makefile && st=1
+test $st -eq 0 || fatal_ "doesn't cover expected code paths"
+
+st=0
+$MAKE check >stdout || st=1
+cat stdout
+cat foo.log
+cat bar.log
+cat baz.log
+cat qux.log
+test $st -eq 0
+grep "^ foofoofoo$" stdout
+grep "^ barbarbar$" stdout
+grep "^ bazbazbaz$" stdout
+grep "^ quxquxqux$" stdout
+$EGREP '(foofoofoo|barbarbar|bazbazbaz|quxquxqux)' *.log && Exit 1
+
+:
cat output
$FGREP '::OOPS::' output && Exit 1 # Possible infinite recursion.
# Check that at least we don't create a botched global log file.
-# FIXME: GNU make currently creates it!
-# test ! -f "$log"
+ test ! -f "$log"
+ if using_gmake; then
+ grep "[Cc]ircular.*dependency" output | $FGREP "$log"
+ test $st -gt 0
+ else
# Look for possible error messages about circular dependencies from
# either make or our own recipes. At least one such a message must
- # be present.
+ # be present. OTOH, some make implementations (e.g., NetBSD's), while
+ # smartly detecting the circular dependency early and diagnosing it,
+ # still exit with a successful exit status (yikes!). So don't check
+ # the exit status of non-GNU make, to avoid spurious failures.
+ # this case.
err_seen=no
for err_rx in \
'circular.* depend' \
break
done
test $err_seen = yes || Exit 1
- # Some make implementations (e.g., NetBSD's), while smartly detecting
- # the circular dependency early and diagnosing it, still exit with a
- # successful exit status (yikes!). Relax our checks not to fail in
- # this case.
- using_gmake && { test $st -gt 0 || Exit 1; }
- :
+ fi
}
: > test-suite.test
parallel_tests=yes
. ./defs || Exit 1
-set -e
-
cat >> configure.in << 'END'
AC_CONFIG_FILES([sub/Makefile])
AC_OUTPUT
$MAKE check DISABLE_HARD_ERRORS='' && Exit 1
cat test-suite.log
-grep '^FAIL: foo\.test .*exit.*99' test-suite.log
+grep '^ERROR: foo$' test-suite.log
cd sub
# The `-e' is wanted here.
DISABLE_HARD_ERRORS='' $MAKE -e check && Exit 1
cat test-suite.log
-grep '^FAIL: bar\.test .*exit.*99' test-suite.log
+grep '^ERROR: bar$' test-suite.log
cd ..
# Check the distributions.
./config.status Makefile
VERBOSE=yes $MAKE check && Exit 1
grep '^FAIL' test-suite.log && Exit 1
-grep '^FAIL: bar\.test .*exit.*99' sub/test-suite.log
+grep '^ERROR: bar$' sub/test-suite.log
echo 'DISABLE_HARD_ERRORS = zardoz' >> sub/Makefile
VERBOSE=yes $MAKE check
cat > Makefile.am << 'END'
TESTS = foo.test
-## Ugly, but required by foo.test. See below.
-TEST_LOG_COMPILER = echo $$$$ > pid && exec 9>&2 && $(SHELL) -x
+## Provide more debugging info.
+TEST_LOG_COMPILER = $(SHELL) -x
+## Required by foo.test; see below.
+AM_TESTS_FD_REDIRECT = 9>&2
END
# This is hacky and ugly, but has the great advantage of avoiding us a lot
# of pain with background processes and related synchronization issues.
+
+cat - "$top_testsrcdir"/lib/test-driver > test-driver <<'END'
+#!/bin/sh
+echo $$ > pid
+END
+
cat > foo.test << 'END'
#!/bin/sh
exec 2>&9
echo "foo is starting to run"
ls -l >&2
-cat foo.log-t >&2 || : > fail
-grep '^foo is starting to run$' foo.log-t >&2 || : > fail
+cat foo.log >&2 || : > fail
+grep '^foo is starting to run$' foo.log >&2 || : > fail
cat pid >&2 || : > fail
kill -$signum `cat pid` || : > fail
END
$ACLOCAL
$AUTOCONF
-$AUTOMAKE -a
+$AUTOMAKE
./configure
trapped_signals='1 2 13 15'
for signum in $trapped_signals; do
- rm -f pid fail *.log *.log-t
+ rm -f pid fail *.log
env signum=$signum $MAKE check && { ls -l; Exit 1; }
ls -l
- cat foo.log-t || :
+ # These files shouldn't exist, but in case they do, their content might
+ # provide helpful information about the causes of the failure(s).
cat foo.log || :
cat test-suite.log || :
test -f fail && Exit 1
- test -f foo.log-t && Exit 1
- test -f foo.log && Exit 1
+ ls | $EGREP 'foo.*\.(log|tmp)' && Exit 1
done
:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2009, 2010, 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check parallel-tests features:
+# - per-extension "test runners" a.k.a. "log compilers" (xxx_LOG_COMPILER
+# and xxx_LOG_FLAGS), also with AC_SUBST'd stuff.
+# See also related test 'parallel-tests-log-compiler-2.test'.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.in << 'END'
+AC_SUBST([acsubst_compiler], [t-compiler])
+AC_CONFIG_LINKS([chk-compiler:chk-compiler])
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+# Note that automake should not match the '/test' part of 'sub/test'
+# as '.test' suffix.
+TESTS = foo.chk bar.test baz bla.t bli.suff sub/test
+
+TEST_EXTENSIONS = .chk .test .t
+
+CHK_LOG_COMPILER = ./chk-compiler
+TEST_LOG_COMPILER = $(SHELL) $(srcdir)/test-compiler
+T_LOG_COMPILER = @acsubst_compiler@
+LOG_COMPILER = @SHELL@ -x $(srcdir)/noext-compiler
+
+CHK_LOG_FLAGS = --chk
+
+AM_TEST_LOG_FLAGS = --am-test
+
+T_LOG_FLAGS = --t
+AM_T_LOG_FLAGS = --am-t
+
+AM_LOG_FLAGS = -a
+LOG_FLAGS = -b
+
+# `t-compiler' is deliberately not distributed; it should be available
+# in PATH also from "make distcheck".
+EXTRA_DIST = $(TESTS) chk-compiler test-compiler noext-compiler
+END
+
+mkdir sub
+
+cat > chk-compiler <<'END'
+#! /bin/sh
+test x"$1" = x"--chk" || {
+ echo "$0: invalid command line: $*" >&2
+ exit 1
+}
+shift
+exec "$@"
+exit 127
+END
+sed 's/--chk/--am-test/' chk-compiler > test-compiler
+
+mkdir bin
+PATH=`pwd`/bin$PATH_SEPARATOR$PATH; export PATH
+
+cat > bin/t-compiler <<'END'
+#! /bin/sh
+if test x"$1,$2" != x"--am-t,--t"; then
+ echo "$0: invalid command line: $*" >&2
+ exit 1
+fi
+shift; shift
+exec "$@"
+exit 127
+END
+sed 's/--am-t,--t/-a,-b/' bin/t-compiler > noext-compiler
+
+chmod a+x chk-compiler chk-compiler test-compiler bin/t-compiler noext-compiler
+
+cat >foo.chk << 'END'
+#! /bin/sh
+exit 0
+END
+chmod a+x foo.chk
+cp foo.chk bar.test
+cp foo.chk baz
+cp foo.chk bla.t
+cp foo.chk bli.suff
+cp foo.chk sub/test
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+./configure
+$MAKE check || { cat test-suite.log; Exit 1; }
+ls -l . sub
+cat test-suite.log
+test -f foo.log
+test -f bar.log
+test -f baz.log
+test -f bla.log
+test -f bli.suff.log
+test -f sub/test.log
+
+T_LOG_FLAGS=--bad $MAKE -e check && Exit 1
+cat test-suite.log
+cat bla.log
+# With the above flag overridden, bla.t should fail ...
+$EGREP '(^ *|/)t-compiler:.* invalid .*--bad' bla.log
+# ... but no other test should.
+grep -v '^FAIL: bla\.t ' bla.log | grep 'FAIL:' && Exit 1
+
+# Try also with a VPATH build.
+$MAKE distcheck
+
+:
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Check parallel-tests features:
-# - per-extension test drivers
+# - per-extension "test runners" a.k.a. "log compilers" (xxx_LOG_COMPILER
+# and xxx_LOG_FLAGS), when some tests are PROGRAMS.
+# See also related test 'parallel-tests-log-compiler-1.test'.
parallel_tests=yes
+required='cc native'
. ./defs || Exit 1
+
cat >> configure.in << 'END'
AC_PROG_CC
AC_OUTPUT
TESTS = foo.chk bar.test $(check_PROGRAMS) sub/test
check_PROGRAMS = baz bla.test bli.suff
TEST_EXTENSIONS = .chk .test
-CHK_LOG_COMPILER = ./chk-driver
-TEST_LOG_COMPILER = ./test-driver
-LOG_COMPILER = ./noext-driver
+CHK_LOG_COMPILER = ./chk-compiler
+TEST_LOG_COMPILER = ./test-compiler
+LOG_COMPILER = ./noext-compiler
AM_CHK_LOG_FLAGS = 1
CHK_LOG_FLAGS = 2
AM_TEST_LOG_FLAGS = 3
mkdir sub
-cat >chk-driver <<'END'
+cat >chk-compiler <<'END'
#! /bin/sh
echo $0 "$@"
shift
exec "$@"
exit 127
END
-chmod a+x chk-driver
-cp chk-driver test-driver
-cp chk-driver noext-driver
+chmod a+x chk-compiler
+cp chk-compiler test-compiler
+cp chk-compiler noext-compiler
cat >foo.chk << 'END'
#! /bin/sh
$AUTOMAKE -a
./configure
+
$MAKE
-$MAKE check
-grep 'chk-driver *1 *2' foo.log
-grep 'test-driver *3 *4' bar.log
-grep 'noext-driver *5 *6' baz.log
-grep 'test-driver *3 *4' bla.log
-grep 'noext-driver *5 *6' bli.suff.log
-grep 'noext-driver *5 *6' sub/test.log
+$MAKE check || { cat test-suite.log; Exit 1; }
+ls -l . sub
+cat test-suite.log
+grep 'chk-compiler *1 *2' foo.log
+grep 'test-compiler *3 *4' bar.log
+grep 'noext-compiler *5 *6' baz.log
+grep 'test-compiler *3 *4' bla.log
+grep 'noext-compiler *5 *6' bli.suff.log
+grep 'noext-compiler *5 *6' sub/test.log
+
:
required=python
. ./defs || Exit 1
-set -e
-
cat >> configure.in <<END
AC_SUBST([PERL], ['$PERL'])
AM_PATH_PYTHON
parallel_tests=yes
. ./defs || Exit 1
-set -e
-
cat >> configure.in <<'END'
AC_OUTPUT
END
parallel_tests=yes
. ./defs || Exit 1
-set -e
-
cat >> configure.in <<'END'
AC_OUTPUT
END
$MAKE -e check >stdout || { cat stdout; Exit 1; }
cat stdout
ls -l
+ count_test_results total=2 pass=1 fail=0 skip=1 xfail=0 xpass=0 error=0
cat pass.log
cat skip.log
cat partial.log
test ! -f pass2.log
test ! -f skip2.log
test ! -f fail.log
- grep '^PASS: .*pass\.test' stdout
- grep '^SKIP: .*skip\.test' stdout
- $FGREP 'skip.test' partial.log
+ grep '^PASS: pass\.test$' stdout
+ grep '^SKIP: skip\.test$' stdout
+ $FGREP 'SKIP: skip' partial.log
$FGREP '% skipped test %' partial.log
- for t in pass2 skip2 fail; do
- $FGREP "$t.test" stdout && Exit 1
- $FGREP "$t.test" partial.log && Exit 1
- done
+ $EGREP '(pass2|skip2|fail)\.test' stdout && Exit 1
+ $EGREP '(pass2|skip2|fail)' partial.log && Exit 1
rm -f *.log
done
parallel_tests=yes
. ./defs || Exit 1
-set -e
-
cat >> configure.in <<'END'
AC_OUTPUT
END
cat > Makefile.am <<'END'
-TESTS = foo.test bar.test baz.test
+TESTS = foofoo.test barbar.test bazbaz.test
END
-cat > foo.test <<'END'
+cat > foofoo.test <<'END'
#! /bin/sh
echo "this is $0"
exit 0
END
-cat > bar.test <<'END'
+cat > barbar.test <<'END'
#! /bin/sh
echo "this is $0"
exit 99
END
-cat > baz.test <<'END'
+cat > bazbaz.test <<'END'
#! /bin/sh
echo "this is $0"
exit ${BAZ_EXIT_STATUS-1}
$AUTOCONF
$AUTOMAKE -a
+# Filter make output before grepping it, for make implementations that,
+# like Solaris' one, print the whole of the failed recipe on failure.
+filter_stdout ()
+{
+ grep -v 'bases=.*;' stdout > t && mv -f t stdout
+}
+
./configure
$MAKE check >stdout && { cat stdout; Exit 1; }
cat stdout
&& { cat stdout; Exit 1; }
cat stdout
ls -l
-grep '^2 of 2 .*failed' stdout
+filter_stdout
+count_test_results total=2 pass=0 fail=1 skip=0 xfail=0 xpass=0 error=1
for x in stdout my.log; do
- $FGREP foo.test $x && Exit 1
- $FGREP bar.test $x
- $FGREP baz.test $x
+ $FGREP foofoo $x && Exit 1
+ $FGREP barbar $x
+ $FGREP bazbaz $x
done
chmod a-rw my.log
&& { cat stdout; Exit 1; }
cat stdout
ls -l
-grep '^1 of 2 .*failed' stdout
-$FGREP foo.test stdout && Exit 1
-$FGREP bar.test stdout
-$FGREP baz.test stdout
-$FGREP foo.test my2.log && Exit 1
-$FGREP bar.test my2.log
-$FGREP baz.test my2.log && Exit 1
+count_test_results total=2 pass=1 fail=0 skip=0 xfail=0 xpass=0 error=1
+filter_stdout
+$FGREP foofoo stdout && Exit 1
+$FGREP barbar stdout
+$FGREP bazbaz stdout
+$FGREP foofoo my2.log && Exit 1
+$FGREP barbar my2.log
+$FGREP bazbaz my2.log && Exit 1
chmod u+r test-suite.log my.log
-$FGREP baz.test test-suite.log
-$FGREP baz.test my.log
+$FGREP bazbaz test-suite.log
+$FGREP bazbaz my.log
:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Colorized output from the testsuite report shouldn't end up in log files.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+esc='\e'
+
+# Check that grep can parse nonprinting characters.
+# BSD 'grep' works from a pipe, but not a seekable file.
+# GNU or BSD 'grep -a' works on files, but is not portable.
+case `echo "$esc" | $FGREP "$esc"` in
+ "$esc") ;;
+# FIXME: creative quoting below to please maintainer-check.
+ *) echo "$me: f""grep can't parse nonprinting characters" >&2; Exit 77;;
+esac
+
+TERM=ansi; export TERM
+
+cat >>configure.in <<END
+AC_OUTPUT
+END
+
+cat >Makefile.am <<'END'
+LOG_COMPILER = $(SHELL)
+AUTOMAKE_OPTIONS = color-tests parallel-tests
+TESTS = pass fail skip xpass xfail error
+XFAIL_TESTS = xpass xfail
+END
+
+# FIXME: creative quoting to please maintainer-check.
+echo exit '0' > pass
+echo exit '0' > xpass
+echo exit '1' > fail
+echo exit '1' > xfail
+echo exit '77' > skip
+echo exit '99' > error
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE --add-missing
+
+./configure
+mv config.log config-log # Avoid possible false positives below.
+AM_COLOR_TESTS=always $MAKE -e check && Exit 1
+$FGREP "$esc" *.log && Exit 1
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2009, 2010, 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check that `:test-results:' directives in test scripts' output doesn't
+# originate spurious results in the testsuite summary.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+TESTS = foo.test bar.test
+END
+
+cat > foo.test <<'END'
+#! /bin/sh
+echo :test-result:XFAIL
+echo :test-result: SKIP
+echo :test-result:ERROR
+exit 0
+END
+cat > bar.test <<'END'
+#! /bin/sh
+echo :test-result: ERROR
+echo :test-result:FAIL
+echo :test-result: XPASS
+exit 0
+END
+chmod a+x foo.test bar.test
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+./configure
+
+st=0
+$MAKE check >stdout || st=$?
+cat stdout
+cat test-suite.log
+cat foo.log
+cat bar.log
+test $st -eq 0 || Exit $st
+
+grep '^:test-result:XFAIL$' foo.log
+grep '^:test-result: SKIP$' foo.log
+grep '^:test-result:FAIL$' bar.log
+grep '^:test-result: XPASS$' bar.log
+
+count_test_results total=2 pass=2 fail=0 skip=0 xfail=0 xpass=0 error=0
+
+:
#! /bin/sh
-# Copyright (C) 1996, 2000, 2002, 2010, 2011 Free Software Foundation,
-# Inc.
+# Copyright (C) 2011 Free Software Foundation, Inc.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Test for bug reported by David A. Swierczek.
-# Another bug with \ in AC_OUTPUT (this time with whitespace).
+# Tests shouldn't be run multiple times by a simple "make check" in a
+# clean directory. An early implementation of the `.trs' intermediate
+# files incurred a similar problem.
+parallel_tests=yes
. ./defs || Exit 1
-cat > configure.in << 'END'
-AC_INIT
-AM_INIT_AUTOMAKE(nonesuch, nonesuch)
-AC_ARG_PROGRAM
-AC_PROG_INSTALL
-AC_PROG_MAKE_SET
-AC_OUTPUT(\
- Makefile \
- zot
-)
+cat >> configure.in << 'END'
+AC_OUTPUT
END
-: > Makefile.am
-: > zot.in
+cat > Makefile.am << 'END'
+TESTS = foo.test
+END
+
+cat > foo.test <<'END'
+#! /bin/sh
+test -f foo.run && Exit 1
+: > foo.run
+END
+chmod a+x foo.test
$ACLOCAL
$AUTOCONF
-$AUTOMAKE
+$AUTOMAKE -a
./configure
-test -f zot || Exit 1
-ls -a | grep '\\' && Exit 1
+$MAKE check
+test -f foo.run # Sanity check.
:
# GNU or BSD 'grep -a' works on files, but is not portable.
case `echo "$esc" | $FGREP "$esc"` in
"$esc") ;;
- *) echo "$me: $FGREP can't parse nonprinting characters" >&2; Exit 77;;
+ *) skip_ "$FGREP can't parse nonprinting characters" ;;
esac
cat >> configure.in << 'END'
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2009, 2010, 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check parallel-tests features:
+# - suffix rules, with PROGRAMS involved
+# See also sister test 'parallel-tests-suffix.test'.
+
+parallel_tests=yes
+required='cc native'
+. ./defs || Exit 1
+
+cat >> configure.in << 'END'
+AC_PROG_CC
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+## Note that automake should not match the '/test' part of 'sub/test' as
+## '.test' suffix, nor the '/chk' part of 'sub/chk' as '.chk' suffix.
+TESTS = $(dist_TESTS) $(check_PROGRAMS)
+dist_TESTS = foo.chk bar.test sub/test
+check_PROGRAMS = baz bla.test bli.suff sub/chk
+TEST_EXTENSIONS = .chk .test
+EXTRA_DIST = $(dist_TESTS)
+END
+
+mkdir sub
+
+cat >foo.chk << 'END'
+#! /bin/sh
+exit 0
+END
+chmod a+x foo.chk
+cp foo.chk bar.test
+cp foo.chk sub/test
+
+cat >baz.c << 'END'
+int main (void)
+{
+ return 0;
+}
+END
+cp baz.c bla.c
+cp baz.c bli.c
+cp baz.c sub/chk.c
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+./configure
+$MAKE check
+ls -l . sub
+test -f foo.log
+test -f bar.log
+test -f baz.log
+test -f bla.log
+test -f bli.suff.log
+test -f sub/chk.log
+test -f sub/test.log
+
+$MAKE distcheck
+
+:
# Check parallel-tests features:
# - suffix rules
+# See also sister test 'parallel-tests-suffix-prog.test'.
parallel_tests=yes
. ./defs || Exit 1
+
cat >> configure.in << 'END'
-AC_PROG_CC
AC_OUTPUT
END
cat > Makefile.am << 'END'
## Note that automake should not match the '/test' part
## of 'sub/test' as '.test' suffix.
-TESTS = foo.chk bar.test $(check_PROGRAMS) sub/test
-check_PROGRAMS = baz bla.test bli.suff
+TESTS = foo.chk bar.test baz bli.suff sub/test
TEST_EXTENSIONS = .chk .test
+EXTRA_DIST = $(TESTS)
END
mkdir sub
END
chmod a+x foo.chk
cp foo.chk bar.test
+cp foo.chk baz
+cp foo.chk bli.suff
cp foo.chk sub/test
-cat >baz.c << 'END'
-int main (void)
-{
- return 0;
-}
-END
-cp baz.c bla.c
-cp baz.c bli.c
-
$ACLOCAL
$AUTOCONF
$AUTOMAKE -a
./configure
-$MAKE
$MAKE check
+ls -l . sub
test -f foo.log
test -f bar.log
test -f baz.log
-test -f bla.log
test -f bli.suff.log
test -f sub/test.log
+
+$MAKE distcheck
+
:
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Check that the testsuite driver copes well with unreadable test logs.
+# Check that the testsuite driver copes well with unreadable `.log'
+# and `.trs' files.
parallel_tests=yes
. ./defs || Exit 1
+: > t
+chmod a-r t && test ! -r t || skip_ "you can still read unreadable files"
+rm -f t
+
cat >> configure.in << 'END'
AC_OUTPUT
END
cat > Makefile.am << 'END'
TESTS = foo.test bar.test
-XFAIL_TESTS = bar.test
END
cat > foo.test << 'END'
#! /bin/sh
+echo foofoofoo
exit 0
END
+
cat > bar.test << 'END'
#! /bin/sh
-exit 1
+echo barbarbar
+exit 77
END
+
chmod a+x foo.test bar.test
$ACLOCAL
$AUTOCONF
$AUTOMAKE -a
-# The testsuite driver will use this variable, so ensure it sanitizes
-# it and do not allow in spurious values from the environment.
-line=PASS; export line
-
./configure
-$MAKE foo.log
-$MAKE bar.log
-chmod a-r foo.log bar.log
-test ! -r foo.log || Exit 77
-$MAKE test-suite.log >stdout && { cat stdout; Exit 1; }
-cat stdout
-grep '^2 of 2 tests failed *$' stdout
-grep '^2 of 2 tests failed\. *$' test-suite.log
+for files in \
+ 'foo.log bar.log' \
+ 'foo.trs bar.trs' \
+ 'foo.trs bar.log' \
+ 'foo.log bar.trs' \
+; do
+ $MAKE check
+ rm -f test-suite.log
+ chmod a-r $files
+ $MAKE test-suite.log || { ls -l; Exit 1; }
+ ls -l
+ grep '^foofoofoo$' foo.log
+ grep '^:test-result: PASS' foo.trs
+ grep '^barbarbar$' bar.log
+ grep '^:test-result: SKIP' bar.trs
+ grep '^SKIP: bar' test-suite.log
+ grep '^barbarbar$' test-suite.log
+ $EGREP ':.*foo|foofoofoo' test-suite.log && Exit 1
+done
:
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Check parallel-tests features:
-# - VERBOSE
-# - clean
-# - TEST_SUITE_LOG
-# - dependencies between tests
-# - TESTS
-# - TEST_LOGS
-# - RECHECK_LOGS
+# Basic checks on parallel-tests support:
+# - console output
+# - log files, and what goes in 'test-suite.log'
+# - make clean
+# - dependencies between tests
+# - TESTS redefinition at runtime
+# - TEST_LOGS redefinition at runtime
+# - RECHECK_LOGS redefinition at runtime
parallel_tests=yes
. ./defs || Exit 1
END
cat > Makefile.am << 'END'
-TEST_SUITE_LOG = mylog.log
TESTS = foo.test bar.test baz.test
XFAIL_TESTS = bar.test
foo.log: bar.log
# foo.test and bar.test sleep to ensure their logs are always strictly newer
# than the logs of their prerequisites, for HP-UX make. The quoting pleases
# maintainer-check.
-cat >>foo.test <<'END'
+cat > foo.test <<'END'
#! /bin/sh
echo "this is $0"
sleep '1'
exit 0
END
-cat >>bar.test <<'END'
+cat > bar.test <<'END'
#! /bin/sh
echo "this is $0"
sleep '1'
exit 99
END
-cat >>baz.test <<'END'
+cat > baz.test <<'END'
#! /bin/sh
echo "this is $0"
exit 1
$MAKE check >stdout && { cat stdout; Exit 1; }
cat stdout
-# There should be two errors: bar.test is a hard error.
-test `grep -c '^FAIL' stdout` -eq 2
-test -f mylog.log
-test `grep -c '^FAIL' mylog.log` -eq 2
+count_test_results total=3 pass=1 fail=1 skip=0 xfail=0 xpass=0 error=1
+test -f test-suite.log
+cat test-suite.log
+test `grep -c '^FAIL:' test-suite.log` -eq 1
+test `grep -c '^ERROR:' test-suite.log` -eq 1
+$EGREP '^(X?PASS|XFAIL|SKIP)' test-suite.log && Exit 1
test -f baz.log
test -f bar.log
test -f foo.log
-# The summary should be formatted correctly.
-grep 'failedn' stdout && Exit 1
-# clean should remove all log files (but not more).
-: > unrelated.log
$MAKE clean
test ! -f baz.log
test ! -f bar.log
test ! -f foo.log
-test ! -f mylog.log
-test -f unrelated.log
+test ! -f test-suite.log
# Check dependencies: baz.test needs to run before bar.test,
# but foo.test is not needed.
# Note that this usage has a problem: the summary will only
# take bar.log into account, because the $(TEST_SUITE_LOG) rule
# does not "see" baz.log. Hmm.
-env TESTS='bar.test' $MAKE -e check && Exit 1
+env TESTS='bar.test' $MAKE -e check >stdout && { cat stdout; Exit 1; }
+cat stdout
+grep '^FAIL: baz\.test$' stdout
+grep '^ERROR: bar\.test$' stdout
+
test -f baz.log
test -f bar.log
test ! -f foo.log
-test -f mylog.log
+test -f test-suite.log
# Upon a lazy rerun, foo.test should be run, but the others shouldn't.
# Note that the lazy rerun still exits with a failure, due to the previous
env RECHECK_LOGS= $MAKE -e check > stdout && { cat stdout; Exit 1; }
cat stdout
test -f foo.log
-grep foo.test stdout
+grep '^PASS: foo\.test$' stdout
grep bar.test stdout && Exit 1
grep baz.test stdout && Exit 1
-grep '2.*tests.*failed' stdout
+grep '^# PASS: *1$' stdout
+grep '^# FAIL: *1$' stdout
+grep '^# ERROR: *1$' stdout
# Now, explicitly retry with all test logs already updated, and ensure
# that the summary is still displayed.
grep foo.test stdout && Exit 1
grep bar.test stdout && Exit 1
grep baz.test stdout && Exit 1
-grep '2.*tests.*failed' stdout
+grep '^# PASS: *1$' stdout
+grep '^# FAIL: *1$' stdout
+grep '^# ERROR: *1$' stdout
# Lazily rerunning only foo should only rerun this one test.
env RECHECK_LOGS=foo.log $MAKE -e check > stdout && { cat stdout; Exit 1; }
grep foo.test stdout
grep bar.test stdout && Exit 1
grep baz.test stdout && Exit 1
-grep '2.*tests.*failed' stdout
-
-# Test VERBOSE.
-env VERBOSE=yes $MAKE -e check > stdout && { cat stdout; Exit 1; }
-cat stdout
-grep 'this is.*bar.test' stdout
-grep 'this is.*baz.test' stdout
+grep '^# PASS: *1$' stdout
+grep '^# FAIL: *1$' stdout
+grep '^# ERROR: *1$' stdout
$MAKE clean
env TEST_LOGS=baz.log $MAKE -e check > stdout && { cat stdout; Exit 1; }
parallel_tests=yes
. ./defs || Exit 1
+
cat >> configure.in << 'END'
AC_OUTPUT
END
./configure
$MAKE check
-Exit 0
+:
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Check parallel-tests features:
-# - check-html
-# - recheck-html
+# - check-html
+# - recheck-html
+# Keep this in sync with sister test `test-driver-custom-html.test'.
parallel_tests=yes
required=rst2html
CLEANFILES = bla
END
-cat >>foo.test <<'END'
+cat > foo.test <<'END'
#! /bin/sh
echo "this is $0"
test -f bla || exit 1
exit 0
END
-cat >>bar.test <<'END'
+
+cat > bar.test <<'END'
#! /bin/sh
echo "this is $0"
exit 99
END
-cat >>baz.test <<'END'
+
+cat > baz.test <<'END'
#! /bin/sh
echo "this is $0"
exit 1
END
+
chmod a+x foo.test bar.test baz.test
$ACLOCAL
$AUTOMAKE -a
./configure
-$MAKE check-html >stdout && { cat stdout; Exit 1; }
-cat stdout
+
+$MAKE check-html && Exit 1
test -f mylog.html
+# check-html should cause check_SCRIPTS to be created.
+test -f bla
+
+# "make clean" should remove HTML files.
+$MAKE clean
+test ! -f mylog.html
+test ! -f bla
# Always create the HTML output, even if there were no failures.
rm -f mylog.html
-env TESTS=foo.test $MAKE -e check-html >stdout || { cat stdout; Exit 1; }
-cat stdout
+env TESTS=foo.test $MAKE -e check-html
test -f mylog.html
-# Create HTML output also with recheck-html
+# Create summarizing HTML output also with recheck-html.
rm -f mylog.html
-env TESTS=foo.test $MAKE -e recheck-html >stdout || { cat stdout; Exit 1; }
-cat stdout
+env TESTS=foo.test $MAKE -e recheck-html
test -f mylog.html
-# check-html and recheck-html should cause check_SCRIPTS to be created,
-# and recheck-html should rerun no tests if check has not been run.
+# check-html should cause check_SCRIPTS to be created.
$MAKE clean
-env TESTS=foo.test $MAKE -e check-html
+env TEST_LOGS=foo.log $MAKE -e check-html
test -f bla
+test -f foo.log
+test -f mylog.html
+# recheck-html should cause check_SCRIPTS to be created, and should rerun
+# no tests if it appears that check has not been run.
$MAKE clean
env TESTS=foo.test $MAKE -e recheck-html
test -f bla
test ! -f foo.log
test -f mylog.html
+
:
required=GNUmake
. ./defs || Exit 1
-# This test does not work well if $MAKE contains -j.
case $MAKE in
-*\ -j*) Exit 77 ;;
+ *\ -j*) skip_ "\$MAKE contains \`-j'";;
esac
cat >> configure.in << 'END'
for i in 1 2 3 4 5 6 7 8; do
echo "TESTS += foo$i.test" >> Makefile.am
- cat >foo$i.test <<'END'
-#! /bin/sh
-echo "this is $0"
-# hack around maintainer-check check:
-sleep='sleep '1
-$sleep
-exit 0
+ unindent >foo$i.test <<'END'
+ #! /bin/sh
+ echo "this is $0"
+ # FIXME: spurious quote to please maintainer-check
+ sleep '1'
+ exit 0
END
chmod a+x foo$i.test
done
cd ..
kill $!
cat parallel/stdout
-test `grep -c PASS parallel/stdout` -eq 8
+test `grep -c '^PASS:' parallel/stdout` -eq 8
# Wait long enough so that there are no open files any more
# when the post-test cleanup runs.
$sleep
done
$sleep
+
:
# It merely serves as demonstration. :-)
parallel_tests=yes
-required=GNUmake
+required='cc native GNUmake'
. ./defs || Exit 1
cat >> configure.in << 'END'
for i in 1 2 3 4 5 6 7 8; do
echo "scripts += foo$i.test" >> Makefile.am
echo "programs += foo$i.prog" >> Makefile.am
- cat >foo$i.test <<'END'
-#! /bin/sh
-echo "this is $0"
-exit 0
+ unindent >foo$i.test <<'END'
+ #! /bin/sh
+ echo "this is $0"
+ exit 0
END
cat >foo$i.c <<'END'
-int main()
-{
- return 0;
-}
+ int main (void)
+ {
+ return 0;
+ }
END
chmod a+x foo$i.test
done
./configure
$MAKE -j4 check
$MAKE distcheck
+
:
./configure
$MAKE check >stdout || { cat stdout; Exit 1; }
cat stdout
-grep '0 tests' stdout
+for x in TOTAL PASS FAIL XPASS FAIL SKIP ERROR; do
+ grep "^# $x: *0$" stdout
+done
Exit 0
DISTCLEANFILES = foo.test
END
-cat >>foo.in <<'END'
+cat > foo.in <<'END'
#! /bin/sh
echo "this is $0"
exit 0
$ACLOCAL
$AUTOCONF
AUTOMAKE_fails -a
-grep '(srcdir.*bar' stderr
-grep 'top_srcdir.*baz' stderr
+grep '$(srcdir).*TESTS.*bar\.test' stderr
+grep '$(top_srcdir).*TESTS.*baz\.test' stderr
sed '/^TESTS +=.*srcdir/d' < Makefile.am > t
mv -f t Makefile.am
echo bla > $@
END
-cat >>foo.test <<'END'
+cat > foo.test <<'END'
#! /bin/sh
echo "this is $0"
test -f bla || exit 1
exit 0
END
-cat >>bar.test <<'END'
+cat > bar.test <<'END'
#! /bin/sh
echo "this is $0"
exit 99
END
-cat >>baz.test <<'END'
+cat > baz.test <<'END'
#! /bin/sh
echo "this is $0"
exit 1
./configure
$MAKE check >stdout && { cat stdout; Exit 1; }
cat stdout
+count_test_results total=3 pass=1 fail=1 skip=0 xfail=0 xpass=0 error=1
$MAKE recheck >stdout && { cat stdout; Exit 1; }
cat stdout
-grep foo.test stdout && Exit 1
-grep bar.test stdout || Exit 1
-grep baz.test stdout || Exit 1
-grep '2 of 2.*failed' stdout
-
-# If we cannot read the log file, then redo it as well.
-chmod a-r foo.log
-if test ! -r foo.log; then
- $MAKE recheck >stdout && { cat stdout; Exit 1; }
- cat stdout
- grep foo.test stdout || Exit 1
-fi
+count_test_results total=2 pass=0 fail=1 skip=0 xfail=0 xpass=0 error=1
+grep 'foo\.test' stdout && Exit 1
+grep '^ERROR: bar\.test$' stdout
+grep '^FAIL: baz\.test$' stdout
# Ensure that recheck builds check_SCRIPTS, and that
# recheck reruns nothing if check has not been run.
test ! -f bar.log
test ! -f baz.log
test -f mylog.log
+
:
--- /dev/null
+# -*- shell-script -*-
+#
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Helper functions used by "plain" tests of the Automake testsuite
+# (i.e., tests that don't use any test protocol).
+
+# Print warnings (e.g., about skipped and failed tests) to this file
+# number. Override by putting, say:
+# AM_TESTS_ENVIRONMENT = stderr_fileno_=9; export stderr_fileno_;
+# AM_TESTS_FD_REDIRECT = 9>&2
+# in your Makefile.am.
+# This is useful when using automake's parallel tests mode, to print the
+# reason for skip/failure to console, rather than to the *.log files.
+: ${stderr_fileno_=2}
+
+# Copied from Gnulib's `tests/init.sh'.
+warn_ () { echo "$@" 1>&$stderr_fileno_; }
+fail_ () { warn_ "$me: failed test: $@"; Exit 1; }
+skip_ () { warn_ "$me: skipped test: $@"; Exit 77; }
+fatal_ () { warn_ "$me: hard error: $@"; Exit 99; }
+framework_failure_ () { warn_ "$me: set-up failure: $@"; Exit 99; }
+
+# For compatibility with TAP functions.
+skip_all_ () { skip_ "$@"; }
+
+:
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Test that POSIX variable expansion `$(var:str=rpl)' works
-# in when used in LDADD.
+# when used in LDADD.
+required=cc
. ./defs || Exit 1
cat >> configure.in << 'END'
libquux1.c:
echo 'int quux1(void) { return 0; }' > $@
-CLEANFILES = libquux1.c # for FreeBSD make
+CLEANFILES = libquux1.c # For FreeBSD make.
END
cat > foo.c <<'END'
# with the LIBRARIES primary in a "simple" way.
# Keep this in sync with sister test `posixsubst-ltlibraries.test'.
+required=cc
. ./defs || Exit 1
cat >> configure.in << 'END'
libbar.c:
echo 'int bar(void) { return 0; }' > $@
-CLEANFILES = libbar.c # for FreeBSD make
+CLEANFILES = libbar.c # For FreeBSD make.
installcheck-local:
ls -l $(prefix)/lib
# with the LTLIBRARIES primary in a "simple" way.
# Keep this in sync with sister test `posixsubst-libraries.test'.
-required='libtool libtoolize'
+required='cc libtool libtoolize'
. ./defs || Exit 1
cat >> configure.in << 'END'
libbar.c:
echo 'int bar(void) { return 0; }' > $@
-CLEANFILES = libbar.c # for FreeBSD make
+CLEANFILES = libbar.c # For FreeBSD make.
installcheck-local:
ls -l $(prefix)/lib
# Test that POSIX variable expansion `$(var:str=rpl)' works when used
# with the PROGRAMS primary.
+required=cc
. ./defs || Exit 1
cat >> configure.in << 'END'
test -f quux.pl
test2: distdir
ls -l $(distdir)
- ## the scripts foo1.sh and foo2.sh should be distributed
+## The scripts foo1.sh and foo2.sh should be distributed.
test -f $(distdir)/foo1.sh
test -f $(distdir)/foo2.sh
- ## the scripts bar1, bar2 and quux.pl shouldn't be distributed
+## The scripts bar1, bar2 and quux.pl shouldn't be distributed.
test ! -r $(distdir)/bar1
test ! -r $(distdir)/bar2
test ! -r $(distdir)/quux.pl
# Test that POSIX variable expansion `$(var:str=rpl)' works when used
# with the SOURCES primary.
+required=cc
. ./defs || Exit 1
cat >> configure.in << 'END'
bar.c baz.c:
echo 'int $@ (void) { return 0; }' | sed 's/\.c //' > $@
CLEANFILES = baz.c
-CLEANFILES += bar.c # for FreeBSD make
+CLEANFILES += bar.c # For FreeBSD make.
.PHONY: test test2
check-local: test1 test2
test -f baz.c
test2: distdir
ls -l $(distdir)
- ## these sources should be distributed ...
+## These sources should be distributed ...
test -f $(distdir)/bar.c
test -f $(distdir)/foo.c
test -f $(distdir)/main.c
- ## ... and this shouldn't
+## ... and this shouldn't.
test ! -r $(distdir)/baz.c
END
foo2.test barz:
(echo '#!/bin/sh' && echo 'exit 0') > $@ && chmod a+x $@
-CLEANFILES = foo2.test barz # for FreeBSD make
+CLEANFILES = foo2.test barz # For FreeBSD make.
# Also try an empty match suffix, to ensure that the `:=' in there is
# not confused by the parser with an unportable assignment operator.
$ACLOCAL
$AUTOCONF
-$AUTOMAKE
+$AUTOMAKE -a
./configure
$MAKE check >out 2>&1 && { cat out; Exit 1; }
# Check to make sure we recognize a Makefile.in, even if post-processed
# and renamed.
+required=cc
. ./defs || Exit 1
cat >configure.in <<END
echo '# Post-processed by post-processor 3.14.' > myMakefile.in
cat myMakefile.old >> myMakefile.in
-./configure || Exit $?
+./configure
test -f .deps/fred.Po || test -f _deps/fred.Po || Exit 1
# For PR 204.
# Sources derived from nodist_ sources should not be distributed.
-required='yacc gcc'
+required='cc yacc'
. ./defs || Exit 1
cat >> configure.in <<'EOF'
# to debug the problem.
# Note that this should be also in the documentation.
-required=gcc
. ./defs || Exit 1
-cat > main.c <<'EOF'
-int main() { return 0; }
-EOF
-
cat > Makefile.am << 'EOF'
if NEVER_TRUE
-NEVER_DEFINED = missingfile.c
+NEVER_DEFINED = foo.txt
endif
-
-bin_PROGRAMS = main
-main_SOURCES = main.c $(NEVER_DEFINED)
+data_DATA = $(NEVER_DEFINED)
EOF
cat >> configure.in << 'EOF'
-AC_PROG_CC
-AC_ARG_ENABLE(foo,
+AC_ARG_ENABLE([foo],
[ --enable-foo Enable foo],
[ if test "foo" = "bar" ; then
- AM_CONDITIONAL(NEVER_TRUE, true)
+ AM_CONDITIONAL([NEVER_TRUE], [true])
else
- AM_CONDITIONAL(NEVER_TRUE, false)
+ AM_CONDITIONAL([NEVER_TRUE], [false])
fi
])
AC_OUTPUT
# `$(top_srcdir)/build', the location of the `.deps' directory
# should be `$(top_builddir)/bar/.deps'.
-required=gcc
+required=cc
. ./defs || Exit 1
mkdir foo
cat >foo/main.c <<'EOF'
-int main() { return 0; }
+int main (void)
+{
+ return 0;
+}
EOF
cat >Makefile.am <<'EOF'
# the AM_OUTPUT_DEPENDENCY_COMMANDS macro cannot cope with
# the colon in the CONFIG_FILES variable.
+required=cc
. ./defs || Exit 1
cat > configure.in << END
$ACLOCAL
$AUTOMAKE
$AUTOCONF
-# FIXME: require compiler in $required rather than skipping here
-./configure || Exit $?
+./configure
test -f .deps/fx64.Po
# Test for PR 266.
# Dependency tracking -vs- nonstandard Makefile names
+required=cc
. ./defs || Exit 1
cat > configure.in << END
$ACLOCAL
$AUTOMAKE
$AUTOCONF
-# FIXME: require compiler in $required rather than skipping here
-./configure --enable-dependency-tracking || Exit $?
+./configure --enable-dependency-tracking
test -f .deps/fx64.Po
. ./defs || Exit 1
cat >> configure.in << 'END'
-AC_PROG_CC
AC_OUTPUT
END
# Make sure that installing subdirectory libraries works.
# PR/300
-required=gcc
+required=cc
. ./defs || Exit 1
cat >> configure.in << 'END'
# Make sure that installing subdirectory libtool libraries works.
# PR/300
-required='libtoolize gcc'
+required='cc libtoolize'
. ./defs || Exit 1
cat >> configure.in << 'END'
int i = 3;
END
-## A rule in the Makefile should create subdir
-# mkdir subdir
-
libtoolize
$ACLOCAL
$AUTOCONF
./configure --prefix "`pwd`/inst"
-$MAKE V=1 >stdout || { cat stdout; Exit 1; }
+# A rule in the Makefile should create subdir.
+test ! -d subdir
+$MAKE >stdout || { cat stdout; Exit 1; }
cat stdout
+test -d subdir
grep 'liba.la .*-rpath .*lib' stdout
grep 'liba.la .*-rpath .*lib/subdir' stdout && Exit 1
# Make sure that installing subdirectory programs works.
# PR/300
-required=gcc
+required=cc
. ./defs || Exit 1
cat >> configure.in << 'END'
: >foo.h
for i in a b c d e f g h; do
- cat >$i.c <<EOF
-#include "foo.h"
-int $i() { return 0; }
+ unindent >$i.c <<EOF
+ #include "foo.h"
+ int $i () { return 0; }
EOF
done
$AUTOMAKE -a
# Sanity check: make sure the variable we are attempting to force
-# is used by configure
+# is indeed used by configure.
grep am_cv_CC_dependencies_compiler_type configure
./configure am_cv_CC_dependencies_compiler_type=dashmstdout
$MAKE check
# The failure we check usually occurs during the above build,
-# with an output such as
+# with an output such as:
#
-# mv -f .libs/f.lo f.lo
-# mv: cannot stat `.libs/f.lo': No such file or directory
+# mv -f .libs/f.lo f.lo
+# mv: cannot stat `.libs/f.lo': No such file or directory
#
# (This may happen on `f' or on some other files.)
test -f librace.la
-test -f tst # a proof that check-local was run
+test -f tst # A proof that check-local was run.
:
# (pr401b.test and pr401c.test do the same for LTLIBOBJS and ALLOCA)
parallel_tests=no
-required=gcc
+required=cc
. ./defs || Exit 1
mkdir lib src
cat >lib/feep.c <<'EOF'
-char *
-feep ()
+const char *feep (void)
{
return "feep";
}
cat >src/main.c <<'EOF'
#include <stdio.h>
-extern char *feep ();
+extern const char *feep (void);
-int
-main (int argc, char **argv)
+int main (void)
{
puts (feep ());
return 0;
AC_LIBSOURCE([feep.c])
AC_PROG_RANLIB
AC_CONFIG_FILES([lib/Makefile src/Makefile])
+AM_CONDITIONAL([CROSS_COMPILING], [test $cross_compiling = yes])
AC_OUTPUT
EOF
check_PROGRAMS = main
main_LDADD = ../lib/libfeep.a
+if !CROSS_COMPILING
TESTS = main
+endif
EOF
$ACLOCAL
check_PROGRAMS = main
main_LDADD = libfeep.a
+if !CROSS_COMPILING
TESTS = main
+endif
EOF
$ACLOCAL
src_main_SOURCES = src/main.c
src_main_LDADD = lib/libfeep.a
+if !CROSS_COMPILING
TESTS = src/main
+endif
check-local:
test -f src/main.$(OBJEXT)
# (pr401.test and pr401c.test do the same for LIBOBJS and ALLOCA)
parallel_tests=no
-required='gcc libtoolize'
+required='cc libtoolize'
. ./defs || Exit 1
mkdir lib src
cat >lib/feep.c <<'EOF'
-char *
-feep ()
+const char *feep (void)
{
return "feep";
}
cat >src/main.c <<'EOF'
#include <stdio.h>
-extern char *feep ();
+extern const char *feep (void);
-int
-main (int argc, char **argv)
+int main (void)
{
puts (feep ());
return 0;
AC_LIBSOURCE([feep.c])
AC_PROG_LIBTOOL
AC_CONFIG_FILES([lib/Makefile src/Makefile])
+AM_CONDITIONAL([CROSS_COMPILING], [test $cross_compiling = yes])
AC_OUTPUT
EOF
check_PROGRAMS = main
main_LDADD = ../lib/libfeep.la
+if !CROSS_COMPILING
TESTS = main
+endif
EOF
libtoolize
./configure
$MAKE distcheck
-
-## ------------------------------------------ ##
-## Traditional LIBOBJS with LIBOBJDIR set. ##
-## ------------------------------------------ ##
+## ----------------------------------------- ##
+## Traditional LTLIBOBJS with LIBOBJDIR set. ##
+## ----------------------------------------- ##
# Invocation of AC_CONFIG_LIBOBJ_DIR may be necessary for reasons
# unrelated to Automake or Makefile.am layout.
check_PROGRAMS = main
main_LDADD = libfeep.la
+if !CROSS_COMPILING
TESTS = main
+endif
EOF
$ACLOCAL
src_main_SOURCES = src/main.c
src_main_LDADD = lib/libfeep.la
+if !CROSS_COMPILING
TESTS = src/main
+endif
check-local:
test -f src/main.$(OBJEXT)
# (pr401.test and pr401b.test do the same for LIBOBJS and LTLIBOBJS)
parallel_tests=no
-required=gcc
+required=cc
. ./defs || Exit 1
mkdir lib src
export ac_cv_func_alloca_works
cat >lib/alloca.c <<'EOF'
-char *
-feep ()
+const char *feep (void)
{
return "feep";
}
cat >src/main.c <<'EOF'
#include <stdio.h>
-extern char *feep ();
+extern const char *feep (void);
-int
-main (int argc, char **argv)
+int main (void)
{
puts (feep ());
return 0;
EOF
cat >>configure.in << 'EOF'
-## These lines are activated for later tests
+## These lines are activated for later tests.
#: AC_CONFIG_LIBOBJ_DIR([lib])
AC_PROG_CC
#x AM_PROG_CC_C_O
AC_PROG_RANLIB
AC_FUNC_ALLOCA
AC_CONFIG_FILES([lib/Makefile src/Makefile])
+AM_CONDITIONAL([CROSS_COMPILING], [test $cross_compiling = yes])
AC_OUTPUT
EOF
check_PROGRAMS = main
main_LDADD = ../lib/libfeep.a
+if !CROSS_COMPILING
TESTS = main
+endif
EOF
$ACLOCAL
$MAKE distcheck
-## ------------------------------------------ ##
-## Traditional LIBOBJS with LIBOBJDIR set. ##
-## ------------------------------------------ ##
+## --------------------------------------- ##
+## Traditional ALLOCA with LIBOBJDIR set. ##
+## --------------------------------------- ##
# Invocation of AC_CONFIG_LIBOBJ_DIR may be necessary for reasons
# unrelated to Automake or Makefile.am layout.
noinst_LIBRARIES = libfeep.a
libfeep_a_SOURCES =
-libfeep_a_LIBADD = $(ALLOCA) $(LIBOBJS) # Add LIBOBJS for fun
+libfeep_a_LIBADD = $(ALLOCA) $(LIBOBJS) # Add LIBOBJS for fun.
check_PROGRAMS = main
main_LDADD = libfeep.a
+if !CROSS_COMPILING
TESTS = main
+endif
EOF
$ACLOCAL
src_main_SOURCES = src/main.c
src_main_LDADD = lib/libfeep.a
+if !CROSS_COMPILING
TESTS = src/main
+endif
check-local:
test -f src/main.$(OBJEXT)
./configure
$MAKE Makefile
-$EGREP 'FOOBAR|zardoz' Makefile && Exit 99 # Sanity check.
+# Sanity check.
+$EGREP 'FOOBAR|zardoz' Makefile && fatal_ 'unexpected AC_SUBST in Makefile'
echo 'AC_SUBST([FOOBAR])' >> configure.in
# Makefile & config.status. To reproduce this race consistently, we
# need the following hackish wrappers.
-save_AUTOCONF=$AUTOCONF
-
cat > aclocal-wrap <<END
#!/bin/sh
set -ex
# aclocal shouldn't use our autoconf wrapper when extracting
# the races from configure.in.
-AUTOCONF='$save_AUTOCONF'; export AUTOCONF
+AUTOCONF='$AUTOCONF'; export AUTOCONF
$ACLOCAL "\$@"
touch -r config.status aclocal.m4
$stat aclocal.m4
set -ex
# automake shouldn't use our autoconf wrapper when extracting
# the races from configure.in.
-AUTOCONF='$save_AUTOCONF'; export AUTOCONF
+AUTOCONF='$AUTOCONF'; export AUTOCONF
$AUTOMAKE "\$@"
touch -r config.status Makefile.in
$stat Makefile.in
# Test for PR automake/87.
# "make distcheck" fails when two source dirs exist.
-required=gcc
+required=cc
. ./defs || Exit 1
subdirs="foo bar"
for i in $subdirs; do
mkdir $i
- cat >$i/$i.c <<EOF
-int main() { return 0; }
+ unindent >$i/$i.c <<EOF
+ int main (void)
+ {
+ return 0;
+ }
EOF
- cat >$i/Makefile.am <<EOF
-bin_PROGRAMS = $i
-${i}_SOURCES = $i.c
+ unindent >$i/Makefile.am <<EOF
+ bin_PROGRAMS = $i
+ ${i}_SOURCES = $i.c
EOF
done
AC_OUTPUT(Makefile `echo $subdirs | sed 's|\([a-z][a-z]*\)|\1/Makefile|g'`)
EOF
-# Ignore user CFLAGS.
-CFLAGS=
-export CFLAGS
-
mkdir build
$ACLOCAL
if test -z "$next"; then
Exit 1
else
- : # for shells with broken 'set -e'
+ : # For shells with broken 'set -e'.
fi
break
fi
tar tf $distdir.tar | $FGREP support
tar xf $distdir.tar
-ls -l . $distdir # for debugging
+ls -l . $distdir # For debugging.
test -d $distdir/support
-ls -l $distdir/support # for debugging
+ls -l $distdir/support # For debugging.
test -f $distdir/support/install-sh
:
. ./defs || Exit 1
-set -e
-
cat >> configure.in <<'END'
AC_PROG_CC
AC_PROG_RANLIB
required=cc
. ./defs || Exit 1
-set -e
-
cat >> configure.in <<'END'
AC_PROG_CC
AC_PROG_RANLIB
. ./defs || Exit 1
-set -e
-
oIFS=$IFS # Saved for later.
: > ltmain.sh
. ./defs || Exit 1
-set -e
-
cat >> configure.in <<'END'
AC_PROG_CC
AC_PROG_RANLIB
# Make sure we can build programs ending in `.la'
-required=gcc
+required=cc
. ./defs || Exit 1
cat >>configure.in <<EOF
cat > Makefile.am << 'END'
bin_PROGRAMS = foo.la
foo_la_SOURCES = foo.c
+.PHONY: test
+test: all
+ test -f foo.la$(EXEEXT)
END
echo 'int main () { return 0; }' > foo.c
$AUTOCONF
$AUTOMAKE
./configure
-$MAKE
+$MAKE test
:
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Make sure a multi-line definition cannot be terminated by an empty
-# line (when there are @substitutions@ inside).
+# Make sure xxx_PROGRAMS gets properly rewritten where needed. These
+# checks have been introduced in commit `Release-1-9-254-g9d0eaef' into
+# the former test `subst2.test'.
+
. ./defs || Exit 1
cat >> configure.in << 'END'
-AC_PROG_CC
-AC_SUBST([ABCDEFGHIJKLMNOPQRSTUVWX])
-AC_SUBST([ABCDEFGHIJKLMNOPQRSTUVWXY])
-AC_SUBST([ABCDEFGHIJKLMNOPQRSTUVWXYZ])
+AC_SUBST([FOO], [c])
AC_OUTPUT
END
cat >Makefile.am <<'END'
-bin_PROGRAMS = x @ABCDEFGHIJKLMNOPQRSTUVWX@ @ABCDEFGHIJKLMNOPQRSTUVWXY@ @ABCDEFGHIJKLMNOPQRSTUVWXYZ@
+AUTOMAKE_OPTIONS = no-dependencies
+CC = false
+EXEEXT = .bin
+
+check_PROGRAMS = a
+bin_PROGRAMS = b @FOO@
+noinst_PROGRAMS = $(bar) $(baz:=de)
EXTRA_PROGRAMS =
-EXEEXT = .bin
+bar = zardoz
+baz = mau
-print-programs:
- @echo BEG: $(bin_PROGRAMS) :END
+.PHONY: test
+test:
+ test '$(check_PROGRAMS)' = a.bin
+ test '$(bin_PROGRAMS)' = 'b.bin c'
+ test '$(noinst_PROGRAMS)' = 'zardoz.bin maude.bin'
END
$ACLOCAL
-$AUTOCONF
$AUTOMAKE
+
+grep PROGRAMS Makefile.in # For debugging.
+
+# Check that no useless indirections are used.
+grep '^check_PROGRAMS = a$(EXEEXT)$' Makefile.in
+grep '^bin_PROGRAMS = b$(EXEEXT) @FOO@$' Makefile.in
+
+$AUTOCONF
./configure
-EXEEXT=.bin $MAKE print-programs >foo
-cat foo
-grep 'BEG: x.bin :END' foo
-EXEEXT=.bin am__empty=X $MAKE -e print-programs >foo
-cat foo
-grep 'BEG: x.bin X :END' foo
-
-# Test for another bug, where EXTRA_PROGRAMS was removed because it was empty.
-grep EXTRA_PROGRAMS Makefile.in
+$MAKE test
+
+:
# Check that python support can work well with virtualenvs.
# This test also works as a mild stress-test on the python support.
-required='python virtualenv'
+required='cc python virtualenv'
. ./defs || Exit 1
# In case the user's config.site defines pythondir or pyexecdir.
CONFIG_SITE=/dev/null; export CONFIG_SITE
# Skip the test if a proper virtualenv cannot be created.
-virtualenv --verbose virtenv || Exit 77
-test -f virtenv/bin/activate || Exit 77
+virtualenv --verbose virtenv && test -f virtenv/bin/activate \
+ || skip_ "coulnd't create python virtual environment"
# Activate the virtualenv.
. ./virtenv/bin/activate
# Sanity check.
-test -n "$VIRTUAL_ENV" || Exit 99
+if test -z "$VIRTUAL_ENV"; then
+ framework_failure_ "can't activate python virtual environment"
+fi
cwd=`pwd`
py_version=`python -c 'import sys; print("%u.%u" % tuple(sys.version_info[:2]))'`
mv -f configure.int configure.in
$ACLOCAL --force
$AUTOCONF --force
-# This one should define PYTHON as : and exit successfully
+# This one should define PYTHON as `:' and exit successfully.
./configure
# Any user setting should be used.
../configure --prefix="/usr"
DESTDIR=$instdir $MAKE -e install
-# Perfunctory test that the files were created
+# Perfunctory test that the files were created.
test -f "$instdir/usr/share/my/my.py"
test -f "$instdir/usr/share/my/my.pyc"
test -f "$instdir/usr/share/my/my.pyo"
-# If DESTDIR has made it into the byte compiled files, fail the test
+# If DESTDIR has made it into the byte compiled files, fail the test.
$FGREP "$instdir" "$instdir/usr/share/my/my.pyo" \
"$instdir/usr/share/my/my.pyc" && Exit 1
# Make sure AM_WITH_REGEX compiles and runs.
+required=cc
. ./defs || Exit 1
cat >> configure.in << 'END'
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2010, 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check that the "all" target triggers rebuilt of outdated Makefiles.
+# See also sister test `remake-all-2.test'.
+
+. ./defs || Exit 1
+
+mkfile=Makefile
+
+magic1=::MagicStringOne::
+magic2=__MagicStringTwo__
+magic3=%%MagicStringThree%%
+
+cat > configure.in <<END
+AC_INIT([$me], [1.0])
+AM_INIT_AUTOMAKE
+AC_CONFIG_FILES([$mkfile sub/$mkfile])
+FOO='$magic1'
+AC_OUTPUT
+END
+
+mkdir sub
+echo SUBDIRS = sub > $mkfile.am
+: > sub/$mkfile.am
+
+$ACLOCAL
+$AUTOMAKE
+
+$EGREP "^all-am:.* $mkfile( |$)" $mkfile.in sub/$mkfile.in
+
+$AUTOCONF
+./configure
+
+$FGREP "$magic1" mkfile.in && Exit 1 # Sanity check.
+
+# Two code paths in configure.am:
+
+# - either a file in $(am__configure_deps) has been updated ...
+$sleep
+echo "AC_SUBST([FOO])" >> configure.in
+$MAKE
+$FGREP FOO $mkfile # For debugging.
+$FGREP $magic1 $mkfile
+
+# - ... or not; i.e., Makefile.am or an included file has.
+$sleep
+echo "# $magic2" >> $mkfile.am
+echo "# $magic3" >> sub/$mkfile.am
+$MAKE
+$FGREP $magic2 $mkfile
+$FGREP $magic3 sub/$mkfile
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2010, 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check that the "all" target triggers rebuilt of outdated makefiles.
+# This handle the case in which the makefiles are not called `Makefile'.
+# See also sister test `remake-all-1.test'.
+
+. ./defs || Exit 1
+
+mkfile=foobar
+
+magic1=::MagicStringOne::
+magic2=__MagicStringTwo__
+magic3=%%MagicStringThree%%
+
+cat > configure.in <<END
+AC_INIT([$me], [1.0])
+AM_INIT_AUTOMAKE
+AC_CONFIG_FILES([$mkfile sub/$mkfile])
+FOO='$magic1'
+AC_OUTPUT
+END
+
+mkdir sub
+
+cat > $mkfile.am <<END
+SUBDIRS = sub
+AM_MAKEFLAGS = -f $mkfile
+END
+
+cat > sub/$mkfile.am <<END
+AM_MAKEFLAGS = -f $mkfile
+END
+
+$ACLOCAL
+$AUTOMAKE
+
+$EGREP "^all-am:.* $mkfile( |$)" $mkfile.in sub/$mkfile.in
+
+$AUTOCONF
+./configure
+
+$FGREP "$magic1" mkfile.in && Exit 1 # Sanity check.
+
+# Two code paths in configure.am:
+
+# - either a file in $(am__configure_deps) has been updated ...
+$sleep
+echo "AC_SUBST([FOO])" >> configure.in
+$MAKE -f $mkfile
+$FGREP FOO $mkfile # For debugging.
+$FGREP $magic1 $mkfile
+
+# - ... or not; i.e., Makefile.am or an included file has.
+$sleep
+echo "# $magic2" >> $mkfile.am
+echo "# $magic3" >> sub/$mkfile.am
+$MAKE -f $mkfile
+$FGREP $magic2 $mkfile
+$FGREP $magic3 sub/$mkfile
+
+:
# <http://lists.gnu.org/archive/html/bug-gnulib/2011-04/msg00005.html>
# for more info).
+required=cc
. ./defs || Exit 1
cat >> configure.in <<'END'
edit_h = sed -e 's|[@]foovar@|@foovar@|g'
foo.h: foo.in.h
$(edit_h) < $(srcdir)/foo.in.h > $@-t
- cat $@-t;: for debugging
+ cat $@-t;: For debugging.
mv -f $@-t $@
EXTRA_DIST = foo.in.h
MOSTLYCLEANFILES = foo.h foo.h-t
# <http://lists.gnu.org/archive/html/bug-gnulib/2011-04/msg00005.html>
# for more info).
+required=cc
. ./defs || Exit 1
cat >> configure.in <<'END'
# <http://lists.gnu.org/archive/html/bug-gnulib/2011-04/msg00005.html>
# for more info).
+required=cc
. ./defs || Exit 1
cat >> configure.in <<'END'
# Do not reject slow dependency extractors: we need dependency tracking.
$srcdir/configure --enable-dependency-tracking
- # Skip the test if dependency tracking couldn't be activated.
- $FGREP 'depmode=none' Makefile && Exit 77
+ if $FGREP 'depmode=none' Makefile; then
+ skip_ "automatic dependency tracking couldn't be activated"
+ fi
$MAKE
ls -l
$sleep
sed -e 's/^\( *override_stdio\)=.*$/\1=false/' $srcdir/macros.m4 > t
- diff $srcdir/macros.m4 t && Exit 99 # sanity check
+ diff $srcdir/macros.m4 t && fatal_ "failed to edit macros.m4"
mv -f t $srcdir/macros.m4
using_gmake || $MAKE Makefile
$sleep
sed -e 's/^\( *override_stdio\)=.*$/\1=:/' $srcdir/macros.m4 > t
- diff $srcdir/macros.m4 t && Exit 99 # sanity check
+ diff $srcdir/macros.m4 t && fatal_ "failed to edit macros.m4"
mv -f t $srcdir/macros.m4
using_gmake || $MAKE Makefile
# Test remake rules when an m4 file gets renamed and *simultaneously*
# an m4 macro in it gets renamed. Kudos to Bruno Haible for thinking
-# about this situation.
+# about this situation. See also related test `acloca22.test'.
. ./defs || Exit 1
#! /bin/sh
-# Copyright (C) 2009 Free Software Foundation, Inc.
+# Copyright (C) 2009, 2011 Free Software Foundation, Inc.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# Check that remake rules works for adding a new subdirectory from a
# pre-existing subdirectory.
+# See also the other similar tests `remake-subdir*.test', and the
+# related test `aclocal5.test'.
. ./defs || Exit 1
-set -e
-
cat >> configure.in <<'END'
m4_include([subdirs.m4])
AC_SUBST([MAGIC], [magic])
# Check that remake rules works from subdirectories, even using
# `GNUmakefile' as makefiles name. This obviously requires GNU
# make.
+# See also the other similar tests `remake-subdir*.test', and the
+# related test `aclocal5.test'
required=GNUmake
. ./defs || Exit 1
-set -e
-
magic1='::MagicString::One::'
magic2='__MagicString__Two__'
. ./defs || Exit 1
-set -e
-
cat > configure.in <<END
AC_INIT([$me], [1.0])
AM_INIT_AUTOMAKE([foreign -Wall -Werror])
# Check that remake rules works from subdirectories, even with non-GNU
# make implementations.
+# See also the other similar tests `remake-subdir*.test', and the
+# related test `aclocal5.test'.
. ./defs || Exit 1
-set -e
-
if using_gmake; then
remake=$MAKE
else
# Check that remake rules works from subdirectories, even when makefiles
# are not named "Makefile".
+# See also the other similar tests `remake-subdir*.test', and the
+# related test `aclocal5.test'.
. ./defs || Exit 1
-set -e
-
magic1='::MagicString::One::'
magic2='__MagicString__Two__'
# configure.ac.
# Keep in sync with sister tests `remake10b.test' and `remake10c.test'.
-required=GNUmake
. ./defs || Exit 1
magic1=::MagicStringOne::
magic2=__MagicStringTwo__
magic3=%%MagicStringThree%%
+if using_gmake; then
+ remake="$MAKE nil"
+else
+ remake="$MAKE Makefile"
+fi
+
cat >> configure.in <<END
m4_include([foo.m4])
AC_SUBST([FINGERPRINT], [my_fingerprint])
echo 'm4_define([my_fingerprint], [BadBadBad])' > foo.m4
-$sleep # so that generated autotools files will be newer than prerequisites
+$sleep # So that generated autotools files will be newer than prerequisites.
$ACLOCAL
$AUTOCONF
$AUTOMAKE
fi
$top_srcdir/configure
- $MAKE # should be no-op
+ $MAKE # Should be a no-op.
$sleep
echo "m4_define([my_fingerprint], [$magic1])" > $top_srcdir/foo.m4
- $MAKE nil
- $FGREP FINGERPRINT Makefile # for debugging
+ $remake
+ $FGREP FINGERPRINT Makefile # For debugging.
$FGREP $magic1 Makefile
$sleep
echo "m4_define([my_fingerprint], [$magic2])" > $top_srcdir/foo.m4
- $MAKE nil
- $FGREP FINGERPRINT Makefile # for debugging
+ $remake
+ $FGREP FINGERPRINT Makefile # For debugging.
$FGREP $magic1 Makefile && Exit 1
$FGREP $magic2 Makefile
$sleep
echo "m4_include([bar.m4])" > $top_srcdir/foo.m4
echo "m4_define([my_fingerprint], [$magic3])" > $top_srcdir/bar.m4
- $MAKE nil
- $FGREP FINGERPRINT Makefile # for debugging
+ $remake
+ $FGREP FINGERPRINT Makefile # For debugging.
$FGREP $magic1 Makefile && Exit 1
$FGREP $magic2 Makefile && Exit 1
$FGREP $magic3 Makefile
$sleep
echo "m4_define([my_fingerprint], [$magic1])" > $top_srcdir/bar.m4
- $MAKE nil
+ $remake
$FGREP $magic2 Makefile && Exit 1
$FGREP $magic3 Makefile && Exit 1
$FGREP $magic1 Makefile
$sleep
echo "m4_define([my_fingerprint], [DummyValue])" > $top_srcdir/foo.m4
+ using_gmake || $remake
$MAKE distcheck
- $FGREP $magic1 Makefile && Exit 1 # sanity check
- $FGREP $magic2 Makefile && Exit 1 # likewise
- $FGREP $magic3 Makefile && Exit 1 # likewise
+ $FGREP $magic1 Makefile && Exit 1 # Sanity check.
+ $FGREP $magic2 Makefile && Exit 1 # Likewise.
+ $FGREP $magic3 Makefile && Exit 1 # Likewise.
$MAKE distclean
# Makefile.am.
# Keep in sync with sister tests `remake10a.test' and `remake10c.test'.
-required=GNUmake
. ./defs || Exit 1
magic1=::MagicStringOne::
magic2=__MagicStringTwo__
magic3=%%MagicStringThree%%
+if using_gmake; then
+ remake="$MAKE nil"
+else
+ remake="$MAKE Makefile"
+fi
+
cat >> configure.in <<'END'
AC_OUTPUT
END
echo 'FINGERPRINT = BadBadBad' > foo.am
-$sleep # so that generated autotools files will be newer than prerequisites
+$sleep # So that generated autotools files will be newer than prerequisites.
$ACLOCAL
$AUTOCONF
$AUTOMAKE
fi
$top_srcdir/configure
- $MAKE # should be no-op
+ $MAKE # Should be a no-op.
$sleep
echo "FINGERPRINT = $magic1" > $top_srcdir/foo.am
- $MAKE nil
- $FGREP FINGERPRINT Makefile # for debugging
+ $remake
+ $FGREP FINGERPRINT Makefile # For debugging.
$FGREP $magic1 Makefile
$sleep
echo "FINGERPRINT = $magic2" > $top_srcdir/foo.am
- $MAKE nil
- $FGREP FINGERPRINT Makefile # for debugging
+ $remake
+ $FGREP FINGERPRINT Makefile # For debugging.
$FGREP $magic1 Makefile && Exit 1
$FGREP $magic2 Makefile
$sleep
echo 'include $(top_srcdir)/bar.am' > $top_srcdir/foo.am
echo "FINGERPRINT = $magic3" > $top_srcdir/bar.am
- $MAKE nil
- $FGREP FINGERPRINT Makefile # for debugging
+ $remake
+ $FGREP FINGERPRINT Makefile # For debugging.
$FGREP $magic1 Makefile && Exit 1
$FGREP $magic2 Makefile && Exit 1
$FGREP $magic3 Makefile
$sleep
echo "FINGERPRINT = $magic1" > $top_srcdir/bar.am
- $MAKE nil
+ $remake
$FGREP $magic2 Makefile && Exit 1
$FGREP $magic3 Makefile && Exit 1
$FGREP $magic1 Makefile
$sleep
echo "FINGERPRINT = DummyValue" > $top_srcdir/foo.am
+ using_gmake || $remake
$MAKE distcheck
- $FGREP $magic1 Makefile && Exit 1 # sanity check
- $FGREP $magic2 Makefile && Exit 1 # likewise
- $FGREP $magic3 Makefile && Exit 1 # likewise
+ $FGREP $magic1 Makefile && Exit 1 # Sanity check.
+ $FGREP $magic2 Makefile && Exit 1 # Likewise.
+ $FGREP $magic3 Makefile && Exit 1 # Likewise.
$MAKE distclean
# aclocal.m4.
# Keep in sync with sister tests `remake10a.test' and `remake10b.test'.
-required=GNUmake
. ./defs || Exit 1
magic1=::MagicStringOne::
magic2=__MagicStringTwo__
magic3=%%MagicStringThree%%
+if using_gmake; then
+ remake="$MAKE nil"
+else
+ remake="$MAKE Makefile"
+fi
+
cat >> configure.in <<END
FINGERPRINT='my_fingerprint'
AC_SUBST([FINGERPRINT])
mkdir m4
echo 'AC_DEFUN([my_fingerprint], [BadBadBad])' > m4/foo.m4
-$sleep # so that generated autotools files will be newer than prerequisites
+$sleep # So that generated autotools files will be newer than prerequisites.
$ACLOCAL -I m4
$AUTOCONF
$AUTOMAKE
fi
$top_srcdir/configure
- $MAKE # should be no-op
+ $MAKE # Should be a no-op.
$sleep
echo "AC_DEFUN([my_fingerprint], [$magic1])" > $top_srcdir/m4/foo.m4
- $MAKE nil
- $FGREP FINGERPRINT Makefile # for debugging
+ $remake
+ $FGREP FINGERPRINT Makefile # For debugging.
$FGREP $magic1 Makefile
$sleep
echo "AC_DEFUN([my_fingerprint], [$magic2])" > $top_srcdir/m4/foo.m4
- $MAKE nil
- $FGREP FINGERPRINT Makefile # for debugging
+ $remake
+ $FGREP FINGERPRINT Makefile # For debugging.
$FGREP $magic1 Makefile && Exit 1
$FGREP $magic2 Makefile
$sleep
echo "m4_include([m4/bar.m4])" > $top_srcdir/m4/foo.m4
echo "AC_DEFUN([my_fingerprint], [$magic3])" > $top_srcdir/m4/bar.m4
- $MAKE nil
- $FGREP FINGERPRINT Makefile # for debugging
+ $remake
+ $FGREP FINGERPRINT Makefile # For debugging.
$FGREP $magic1 Makefile && Exit 1
$FGREP $magic2 Makefile && Exit 1
$FGREP $magic3 Makefile
$sleep
echo "AC_DEFUN([my_fingerprint], [$magic1])" > $top_srcdir/m4/bar.m4
- $MAKE nil
+ $remake
$FGREP $magic2 Makefile && Exit 1
$FGREP $magic3 Makefile && Exit 1
$FGREP $magic1 Makefile
$sleep
echo "AC_DEFUN([my_fingerprint], [DummyValue])" > $top_srcdir/m4/foo.m4
echo "AC_DEFUN([AM_UNUSED], [NoSuchMacro])" > $top_srcdir/m4/bar.m4
- $MAKE nil # required to work around limitation of GNU make 3.82
+ using_gmake || $remake
$MAKE distcheck
- $FGREP $magic1 Makefile && Exit 1 # sanity check
- $FGREP $magic2 Makefile && Exit 1 # likewise
- $FGREP $magic3 Makefile && Exit 1 # likewise
+ $FGREP $magic1 Makefile && Exit 1 # Sanity check.
+ $FGREP $magic2 Makefile && Exit 1 # Likewise.
+ $FGREP $magic3 Makefile && Exit 1 # Likewise.
$MAKE distclean
# rules don't break in obvious ways in a slightly "heavier than usual"
# setup.
-required=GNUmake
. ./defs || Exit 1
ocwd=`pwd` || Exit 1
bar_in_list=`find . -name bar.in | LC_ALL=C sort`
bar_list=`echo "$bar_in_list" | sed 's/\.in$//'`
-cat configure.in # useful for debugging
+cat configure.in # For debugging.
$ACLOCAL
$AUTOMAKE
# Test basic remake rules for Makefiles with non-default names
# and/or with multiple sources.
-required=GNUmake
. ./defs || Exit 1
magic1=::MagicStringOne::
magic2=__MagicStringTwo__
magic3=%%MagicStringThree%%
+# See if the user's make implementation support an include directive.
+# We need it in this test.
+echo 'all:; @echo include is supported' > incl.mk
+echo 'include incl.mk' > gnu-style.mk
+echo '.include "incl.mk"' > bsd-style.mk
+if $MAKE -f gnu-style.mk | grep 'include is supported'; then
+ echo "$me: GNU make include style supported"
+ include_zardoz='include zardoz'
+elif $MAKE -f bsd-style.mk | grep 'include is supported'; then
+ echo "$me: BSD make include style supported"
+ include_zardoz='.include "zardoz"'
+else
+ skip_ "make doesn't support any \"include\" directive"
+fi
+
cat > configure.in <<END
AC_INIT([$me], [1.0])
AM_INIT_AUTOMAKE
END
cat > Makefile <<END
-include zardoz
+$include_zardoz
nil:
.PHONY: nil
END
./configure
-$MAKE nil
-grep '^#H:' zardoz.in # for debugging
+if using_gmake; then
+ $MAKE nil
+else
+ $MAKE Makefile
+fi
+grep '^#H:' zardoz.in # For debugging.
$FGREP $magic1 zardoz
$FGREP $magic1 zardoz.in
$MAKE distcheck
-$MAKE distclean # this shouldn't remove Makefile
+$MAKE distclean # This shouldn't remove Makefile.
ls -l
test -f Makefile
$sleep
sed "s/%MAGIC3%/$magic3/" >> Makefile <<'END'
my-check:
- ls -l . $(srcdir) ;: for debugging
+ ls -l . $(srcdir) ;: For debugging.
test -f $(srcdir)/quux.am
test -f $(srcdir)/quux.in
test -f $(srcdir)/bot.in
test -f $(srcdir)/top.in
test ! -r $(srcdir)/zardoz.am
test ! -r $(srcdir)/zardoz.in
- grep FOO zardoz ;: for debugging
+ grep FOO zardoz ;: For debugging.
test x'$(FOO)' = x'%MAGIC3%'
test:
- ls -l ;: for debugging
+ ls -l ;: For debugging.
test x'$(FOO)' = x'dummy'
.PHONY: test my-check
END
sed "s/^#H:.*/#H: $magic2/" zardoz.am > t
cat >> t <<'END'
-# used by "make distcheck" below
+# Used by "make distcheck" below.
check-local: my-check
END
mv -f t zardoz.am
-cat zardoz.am # for debugging
-$MAKE nil
-$FGREP my-check zardoz # sanity check
+cat zardoz.am # For debugging.
+if using_gmake; then
+ $MAKE nil
+else
+ $MAKE zardoz
+fi
+$FGREP my-check zardoz # Sanity check.
$FGREP $magic1 zardoz zardoz.in && Exit 1
$FGREP $magic2 zardoz
$FGREP $magic2 zardoz.in
sed 's/^\(AC_CONFIG_FILES\)(.*/\1([zardoz:top.in:quux.in:bot.in])/' \
<configure.in >t
mv -f t configure.in
-cat configure.in # for debugging
+cat configure.in # For debugging.
sed '/^#H:/d' zardoz.am > quux.am
echo 'FOO = dummy' >> quux.am
echo 'BAR = $(BAZ)' > top.in
echo "BAZ = $magic3" > bot.in
+is_gmake || $MAKE zardoz
$MAKE test
-$FGREP my-check zardoz # sanity check
+$FGREP my-check zardoz # Sanity check.
$FGREP $magic3 quux.in && Exit 1
$FGREP $magic3 zardoz
$FGREP $magic1 zardoz && Exit 1
$FGREP $magic2 zardoz && Exit 1
# After the remake above, the files `zardoz.am' and `zardoz.in'
-# should be no more needed
-echo 'endif' > zardoz.am # put in syntax error
+# should be no more needed.
+echo 'endif' > zardoz.am # Put in a syntax error.
+is_gmake || $MAKE zardoz
$MAKE test
-rm -f zardoz.in zardoz.am # get rid of them
+rm -f zardoz.in zardoz.am # Get rid of them.
+is_gmake || $MAKE zardoz
$MAKE test
echo 'FOO = $(BAR)' >> quux.am
+is_gmake || $MAKE zardoz
$MAKE distcheck
:
# Make sure remaking rules in subdir are correctly generated.
# See also sister "grepping" test `remake.test'.
-required=GNUmake
. ./defs || Exit 1
fingerprint='=/FiNgErPrInT/='
# Rebuild rules should work even if there is no top-level
# makefiles, if we are careful enough to trigger them only
-# in proper subdirs
+# in proper subdirs.
$sleep
echo "# $fingerprint" > sub/Makefile.am
cd sub
cd ..
# Now try an in-tree build.
-: > sub/foo.in # restore its original (empty) content
+: > sub/foo.in # Restore its original (empty) content.
./configure
$sleep
test -f sub/foo
grep '%GrepMe%' sub/foo
END
$MAKE
-grep "grep '%GrepMe%' sub/foo" Makefile.in # sanity check
+grep "grep '%GrepMe%' sub/foo" Makefile.in # Sanity check.
$MAKE distcheck
:
# but not all.
# Report from Ralf Corsepius.
-required=GNUmake
. ./defs || Exit 1
+if using_gmake; then
+ remake="$MAKE"
+else
+ remake="$MAKE Makefile"
+fi
+
cat >>configure.in <<'EOF'
AM_MAINTAINER_MODE
m4_include([foo.m4])
# tools are set to false.)
$sleep
touch aclocal.m4 Makefile.am configure.in foo.m4
-$MAKE
+$remake
# Make sure the rebuild rule for Makefile is triggered.
$sleep
echo '# GrEpMe' >>Makefile.in
-$MAKE
+$remake
grep GrEpMe Makefile
# Make sure the rebuild rule for config.status is triggered.
grep 'AUTOCONF.*=.*false' Makefile
: > rebuild_ok
./configure --no-create
-$MAKE
+$remake
grep 'AUTOCONF.*=.*false' Makefile && Exit 1
# Make sure rebuild rules do work if --enable-maintainer-mode is given.
./configure --enable-maintainer-mode
$sleep
echo 'AC_SUBST([YIPPY_YIPPY_YEAH])' >foo.m4
-$MAKE
+$remake
grep YIPPY_YIPPY_YEAH Makefile
# Try the distribution, for completeness.
# Keep this in sync with sister test `remake8b.test', which performs the
# same checks for a VPATH build.
-required=GNUmake
. ./defs || Exit 1
mv -f configure.in configure.stub
$AUTOCONF
$AUTOMAKE
-srcdir='.' # to make syncing with remake8b.test easier.
+srcdir='.' # To make syncing with remake8b.test easier.
$srcdir/configure
CLEANFILES = bar
END
+using_gmake || $MAKE Makefile
$MAKE
cat bar
grep '!Baz!' bar
$sleep
cat > $srcdir/Makefile.am <<'END'
-all-local: quux
check-local:
cat quux
grep '!Zardoz!' quux
!@QUUX@!
END
+using_gmake || $MAKE Makefile
$MAKE
cat quux
grep '!Zardoz!' quux
cat > $srcdir/Makefile.am <<'END'
ACLOCAL_AMFLAGS = -I m4
-all-local: quux
check-local:
cat quux
grep '%Foo%' quux
test x'$(QUUX)' = x'%Foo%'
END
-$MAKE # this should place aclocal flags in Makefile
-grep '.*-I m4' Makefile # sanity check
+$MAKE # This should place aclocal flags in Makefile.
+grep '.*-I m4' Makefile # Sanity check.
# Modify configure.in and aclocal.m4.
# Keep this in sync with sister test `remake8a.test', which performs the
# same checks for a in-tree build.
-required=GNUmake
. ./defs || Exit 1
mv -f configure.in configure.stub
mkdir build
cd build
-srcdir='..' # to make syncing with remake8a.test easier.
+srcdir='..' # To make syncing with remake8a.test easier.
$srcdir/configure
CLEANFILES = bar
END
+using_gmake || $MAKE Makefile
$MAKE
cat bar
grep '!Baz!' bar
$sleep
cat > $srcdir/Makefile.am <<'END'
-all-local: quux
check-local:
cat quux
grep '!Zardoz!' quux
!@QUUX@!
END
+using_gmake || $MAKE Makefile
$MAKE
cat quux
grep '!Zardoz!' quux
cat > $srcdir/Makefile.am <<'END'
ACLOCAL_AMFLAGS = -I m4
-all-local: quux
check-local:
cat quux
grep '%Foo%' quux
test x'$(QUUX)' = x'%Foo%'
END
-$MAKE # this should place aclocal flags in Makefile
-grep '.*-I m4' Makefile # sanity check
+$MAKE # This should place aclocal flags in Makefile.
+grep '.*-I m4' Makefile # Sanity check.
# Modify configure.in and aclocal.m4.
# Test remake rules when configure.in or its prerequisites change.
# Keep in sync with the other sister tests `remake9*.test'.
-required=GNUmake
. ./defs || Exit 1
magic1=::MagicStringOne::
magic2=__MagicStringTwo__
+if using_gmake; then
+ remake_() { $MAKE nil; }
+else
+ remake_() { $MAKE Makefile && $MAKE foo.sh; }
+fi
+
cat >> configure.in <<END
FINGERPRINT=BadBadBad
AC_SUBST([FINGERPRINT])
echo '@FINGERPRINT@'
END
-echo cat > tweak-configure-in # no-op by default
+echo cat > tweak-configure-in # It is a no-op by default.
-$sleep # so that generated autotools files will be newer than prerequisites
+$sleep # So that generated autotools files will be newer than prerequisites.
$ACLOCAL
$AUTOCONF
$AUTOMAKE
fi
$srcdir/configure
- $MAKE # should be no-op
+ $MAKE # Should be a no-op.
$sleep
sed "s/^\\(FINGERPRINT\\)=.*/\\1=$magic1/" $srcdir/configure.in >t
mv -f t $srcdir/configure.in
- $MAKE nil
- $FGREP FINGERPRINT Makefile # for debugging
+ remake_
+ $FGREP FINGERPRINT Makefile # For debugging.
$FGREP $magic1 Makefile
test x"`./foo.sh`" = x"$magic1"
$sleep
echo 'sed "s/^\\(FINGERPRINT\\)=.*/\\1='$magic2'/"' \
> $srcdir/tweak-configure-in
- $MAKE nil
- $FGREP FINGERPRINT Makefile # for debugging
+ remake_
+ $FGREP FINGERPRINT Makefile # For debugging.
$FGREP $magic1 Makefile && Exit 1
$FGREP $magic2 Makefile
test x"`./foo.sh`" = x"$magic2"
$sleep
- echo cat > $srcdir/tweak-configure-in # no-op again
+ echo cat > $srcdir/tweak-configure-in # Make it a no-op again.
sed "s/^\\(FINGERPRINT\\)=.*/\\1=DummyValue/" $srcdir/configure.in >t
mv -f t $srcdir/configure.in
+ using_gmake || remake_
$MAKE distcheck
- $FGREP $magic1 Makefile && Exit 1 # sanity check
- $FGREP $magic2 Makefile && Exit 1 # likewise
+ $FGREP $magic1 Makefile && Exit 1 # Sanity check.
+ $FGREP $magic2 Makefile && Exit 1 # Likewise.
$MAKE distclean
# Test remake rules when Makefile.am or its prerequisites change.
# Keep in sync with the other sister tests `remake9*.test'.
-required=GNUmake
. ./defs || Exit 1
+if using_gmake; then
+ remake_() { $MAKE nil; }
+else
+ remake_() { $MAKE Makefile && $MAKE foo.sh; }
+fi
+
magic1=::MagicStringOne::
magic2=__MagicStringTwo__
test x"`./foo.sh`" = x"DummyValue"
END
-echo cat > tweak-makefile-am # no-op by default
+echo cat > tweak-makefile-am # It is a no-op by default.
-$sleep # so that generated autotools files will be newer than prerequisites
+$sleep # So that generated autotools files will be newer than prerequisites.
$ACLOCAL
$AUTOCONF
$AUTOMAKE
fi
$srcdir/configure
- $MAKE # should be no-op
+ $MAKE # Should be a no-op.
$sleep
sed "s/^\\(FINGERPRINT\\) *=.*/\\1 = $magic1/" $srcdir/Makefile.am >t
mv -f t $srcdir/Makefile.am
- $MAKE nil
- $FGREP FINGERPRINT Makefile # for debugging
+ remake_
+ $FGREP FINGERPRINT Makefile # For debugging.
$FGREP $magic1 Makefile
test x"`./foo.sh`" = x"$magic1"
$sleep
echo 'sed "s/^\\(FINGERPRINT\\) *=.*/\\1 = '$magic2'/"' \
> $srcdir/tweak-makefile-am
- $MAKE nil
- $FGREP FINGERPRINT Makefile # for debugging
+ remake_
+ $FGREP FINGERPRINT Makefile # For debugging.
$FGREP $magic1 Makefile && Exit 1
$FGREP $magic2 Makefile
test x"`./foo.sh`" = x"$magic2"
$sleep
- echo cat > $srcdir/tweak-makefile-am # no-op again
+ echo cat > $srcdir/tweak-makefile-am # Make it a no-op again.
sed "s/^\\(FINGERPRINT\\) *=.*/\\1 = DummyValue/" $srcdir/Makefile.am >t
mv -f t $srcdir/Makefile.am
+ using_gmake || remake_
$MAKE distcheck
- $FGREP $magic1 Makefile && Exit 1 # sanity check
- $FGREP $magic2 Makefile && Exit 1 # likewise
+ $FGREP $magic1 Makefile && Exit 1 # Sanity check.
+ $FGREP $magic2 Makefile && Exit 1 # Likewise.
$MAKE distclean
# Test remake rules when acinclude.m4 or its prerequisites change.
# Keep in sync with the other sister tests `remake9*.test'.
-required=GNUmake
. ./defs || Exit 1
magic1=::MagicStringOne::
magic2=__MagicStringTwo__
+if using_gmake; then
+ remake_() { $MAKE nil; }
+else
+ remake_() { $MAKE Makefile && $MAKE foo.sh; }
+fi
+
cat >> configure.in <<END
FINGERPRINT='my_fingerprint'
AC_SUBST([FINGERPRINT])
echo "AC_DEFUN([my_fingerprint], [BadBadBad])" > acinclude.m4
-echo : > tweak-acinclude-m4 # no-op by default
+echo : > tweak-acinclude-m4 # It is a no-op by default.
-$sleep # so that generated autotools files will be newer than prerequisites
+$sleep # So that generated autotools files will be newer than prerequisites.
$ACLOCAL
$AUTOCONF
$AUTOMAKE
fi
$srcdir/configure
- $MAKE # should be no-op
+ $MAKE # Should be a no-op.
$sleep
echo "AC_DEFUN([my_fingerprint], [$magic1])" > $srcdir/acinclude.m4
- $MAKE nil
- $FGREP FINGERPRINT Makefile # for debugging
+ remake_
+ $FGREP FINGERPRINT Makefile # For debugging.
$FGREP $magic1 Makefile
test x"`./foo.sh`" = x"$magic1"
$sleep
echo "echo 'AC_DEFUN([my_fingerprint], [$magic2])'" \
> $srcdir/tweak-acinclude-m4
- $MAKE nil
- $FGREP FINGERPRINT Makefile # for debugging
+ remake_
+ $FGREP FINGERPRINT Makefile # For debugging.
$FGREP $magic1 Makefile && Exit 1
$FGREP $magic2 Makefile
test x"`./foo.sh`" = x"$magic2"
$sleep
- echo : > $srcdir/tweak-acinclude-m4 # no-op again
+ echo : > $srcdir/tweak-acinclude-m4 # Make it a no-op again.
echo "AC_DEFUN([my_fingerprint], [DummyValue])" > $srcdir/acinclude.m4
+ using_gmake || remake_
$MAKE distcheck
- $FGREP $magic1 Makefile && Exit 1 # sanity check
- $FGREP $magic2 Makefile && Exit 1 # likewise
+ $FGREP $magic1 Makefile && Exit 1 # Sanity check.
+ $FGREP $magic2 Makefile && Exit 1 # Likewise.
$MAKE distclean
# Test remake rules when aclocal.m4 or its prerequisites change.
# Keep in sync with the other sister tests `remake9*.test'.
-required=GNUmake
. ./defs || Exit 1
magic1=::MagicStringOne::
magic2=__MagicStringTwo__
+if using_gmake; then
+ remake_() { $MAKE nil; }
+else
+ remake_() { $MAKE Makefile && $MAKE foo.sh; }
+fi
+
cat >> configure.in <<END
FINGERPRINT='my_fingerprint'
AC_SUBST([FINGERPRINT])
echo '@FINGERPRINT@'
END
-echo cat > tweak-aclocal-m4 # no-op by default
+echo cat > tweak-aclocal-m4 # It is a no-op by default.
-$sleep # so that generated autotools files will be newer than prerequisites
+$sleep # So that generated autotools files will be newer than prerequisites.
$ACLOCAL
# Cheatingly mark aclocal.m4 as non auto-generated.
sed '/^ *#.*generated automatically/d' aclocal.m4 > t
fi
$srcdir/configure
- $MAKE # should be no-op
+ $MAKE # Should be a no-op.
$sleep
echo "AC_DEFUN([my_fingerprint], [$magic1])dnl %%%" >> $srcdir/aclocal.m4
- $MAKE nil
- $FGREP FINGERPRINT Makefile # for debugging
+ remake_
+ $FGREP FINGERPRINT Makefile # For debugging.
$FGREP $magic1 Makefile
test x"`./foo.sh`" = x"$magic1"
$sleep
echo "sed 's/.*dnl *%%%.*/AC_DEFUN([my_fingerprint], [$magic2])/'" \
> $srcdir/tweak-aclocal-m4
- $MAKE nil
- $FGREP FINGERPRINT Makefile # for debugging
+ remake_
+ $FGREP FINGERPRINT Makefile # For debugging.
$FGREP $magic1 Makefile && Exit 1
$FGREP $magic2 Makefile
test x"`./foo.sh`" = x"$magic2"
$sleep
- echo cat > $srcdir/tweak-aclocal-m4 # no-op again
+ echo cat > $srcdir/tweak-aclocal-m4 # Make it a no-op again.
echo 'AC_DEFUN([my_fingerprint], [DummyValue])' >> $srcdir/aclocal.m4
+ using_gmake || remake_
$MAKE distcheck
- $FGREP $magic1 Makefile && Exit 1 # sanity check
- $FGREP $magic2 Makefile && Exit 1 # likewise
+ $FGREP $magic1 Makefile && Exit 1 # Sanity check.
+ $FGREP $magic2 Makefile && Exit 1 # Likewise.
$MAKE distclean
# Check that automake does not complain on repeated options, nor
# generate broken or incorrect makefiles.
-required=bzip2
+required='cc bzip2'
. ./defs || Exit 1
cat >configure.in <<END
}
END
-cp $testsrcdir/../lib/compile .
+cp "$top_testsrcdir"/lib/compile "$top_testsrcdir"/lib/test-driver .
$ACLOCAL
$AUTOMAKE --foreign --foreign -Wall 2>stderr || { cat stderr >&2; Exit 1; }
$ACLOCAL
AUTOMAKE_fails
-$FGREP CC stderr
-$FGREP CXX stderr
+grep ' C source.*CC.* undefined' stderr
+grep 'define .*CC.* add .*AC_PROG_CC' stderr
+grep ' C++ source.*CXX.* undefined' stderr
+grep 'define .*CXX.* add .*AC_PROG_CXX' stderr
+
+:
. ./defs || Exit 1
if test x"$sh_errexit_works" != x"yes"; then
- echo "$me: the shell can't have a working exit trap with 'set -e'" >&2
- Exit 77
+ skip_ "$me: no working exit trap with 'set -e'"
fi
# We still need a little hack to make ./defs work outside automake's
# tree `tests' subdirectory. Not a big deal.
sed "s|^testbuilddir=.*|testbuilddir='`pwd`'|" ../defs-static >defs-static
+diff ../defs-static defs-static && Exit 99
cp ../defs .
+AM_TESTS_REEXEC=no; export AM_TESTS_REEXEC
+
have_symlinks=false
ln -s defs foo && have_symlinks=:
export have_symlinks # Is used also by spawned shells.
# Check that post-test cleanup works also with directories with
# "null" permissions, and containing broken symlinks.
$SHELL -c '
+ stderr_fileno_=2
. ./defs || Exit 1
set -e
mkdir dir dir/sub
$SHELL -c '
ocwd=`pwd` || exit 1
+ stderr_fileno_=2
. ./defs || Exit 1
ln -s "$ocwd/dir" "$ocwd/file" .
' dummy.test
'kill -13 $$' \
'kill -15 $$' \
; do
- $SHELL -c ". ./defs; : > foo; $bailout_command" dummy.test && Exit 1
+ $SHELL -c "
+ stderr_fileno_=2
+ . ./defs
+ : > foo
+ $bailout_command
+ " dummy.test && Exit 1
test -f dummy.dir/foo
done
# Sanity check for the automake testsuite.
# Check that tests using `./defs' create a proper temporary directory,
-# and run in it.
+# and run in it. Also check that we can prevent the pre-population
+# and the creation of such directory if we want.
. ./defs-static || exit 1
set -ex
+AM_TESTS_REEXEC=no; export AM_TESTS_REEXEC
-this=S_dir
-outcome=:
-pwd
+# This needs to be consistent with what $SHELL deems to be the
+# current working directory.
+cwd=`$SHELL -c 'pwd'` || exit 99
+echo "$cwd"
-rm -f $this.tmp
+rm -f _self.tmp
-$SHELL -c "
+st=0
+$SHELL -c '
. ./defs || Exit 1
- : > $this.tmp
- case \`pwd\` in */$this.dir);; *) Exit 1;; esac
-" $this.test || outcome=false
+ pwd; ls -l; # For debugging.
+ : > _self.tmp
+ test -f install-sh
+ test -f configure.in
+ case `pwd` in '"$cwd"'/_self.dir);; *) Exit 1;; esac
+' _self.test || st=1
+test -f _self.tmp && { rm -f _self.tmp; st=1; }
+test $st -eq 0 || exit 1
-test -f $this.tmp && { rm -f $this.tmp; outcome=false; }
+st=0
+$SHELL -c '
+ am_create_testdir=empty
+ . ./defs || Exit 1
+ pwd; ls -l; # For debugging.
+ ls -a | grep -v "^\\.\\{1,2\\}$" | grep . && Exit 1
+ : > _self.tmp
+ case `pwd` in '"$cwd"'/_self.dir);; *) Exit 1;; esac
+ :
+' _self.test || st=1
+test -f _self.tmp && { rm -f _self.tmp; st=1; }
+test $st -eq 0 || exit 1
+
+$SHELL -c '
+ am_create_testdir=no
+ . ./defs || Exit 1
+ test -f self-check-dir.test
+ case `pwd` in '"$cwd"');; *) Exit 1;; esac
+' _self.test || exit 1
-$outcome
+:
# Make sure that the testsuite initialization code complains when
# some testsuite-influential variables are set in the environment.
-. ./defs-static || exit 1
+am_create_testdir=no
+. ./defs || Exit 1
set -x
exec 5>&1
-for var in me parallel_tests required original_AUTOMAKE original_ACLOCAL; do
- env "$var=foo" $SHELL -c '. ./defs' foo.test && exit 1
- env "$var=foo" $SHELL -c '. ./defs' foo.test 2>&1 1>&5 \
- | grep "foo\.test:.* variable \`$var'.* in the environment.*unsafe" || exit 1
+plan_ 18 # Two times the number of variable names in $vars.
+
+vars='
+ me
+ parallel_tests
+ required
+ using_tap
+ am_create_testdir
+ am_tap_implementation
+ test_prefer_config_shell
+ original_AUTOMAKE
+ original_ACLOCAL
+'
+
+do_run ()
+{
+ env "$1=foo" $SHELL -c '. ./defs' foo.test
+}
+
+do_grep ()
+{
+ env "$1=foo" $SHELL -c '. ./defs' foo.test 2>&1 1>&5 \
+ | grep "foo\.test:.* variable \`$1'.* in the environment.*unsafe"
+}
+
+for var in $vars; do
+ command_ok_ "$var [err status]" not do_run $var
+ command_ok_ "$var [err message]" do_grep $var
done
:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Sanity check for the automake testsuite.
+# Check that, in case of failing commands, the correct exit status is
+# passed to the exit trap installed by the `./defs' script.
+# Also check that the `errexit' shell flag is active.
+
+am_create_testdir=no
+. ./defs || Exit 99
+
+plan_ 34
+
+# This test becomes more cumbersome if we keep the `errexit' shell flag
+# set. And removing it is no big deal, as this test is a TAP-based one,
+# so that false positives remain very unlikely.
+set +e
+
+AM_TESTS_REEXEC=no; export AM_TESTS_REEXEC
+
+init='. ./defs; stderr_fileno_=2; unset am_explicit_skips'
+
+for st in 1 2 3 4 5 77 99 126 127 128 129 130 255; do
+ for exit_cmd in "Exit $st" "sh -c 'exit $st'"; do
+ $SHELL -c "$init; $exit_cmd; :"
+ command_ok_ "$exit_cmd" test $? -eq $st
+ done
+done
+
+for sig in 1 2 13 15; do
+ if test $sig -eq 2; then
+ # Some Korn shells might otherwise get a spurious SIGINT
+ # signal when one is sent to the child $SHELL.
+ trap : 2
+ fi
+ $SHELL -c "$init; kill -$sig \$\$; :"
+ rc=$?
+ if test $sig -eq 2; then
+ # Reset default SIGINT handler as portably as possible.
+ trap 2 || trap - 2
+ fi
+ if test x"$sh_errexit_works" = x"yes"; then
+ # The exit trap should turn into an hard errors any failure
+ # caused by signals.
+ command_ok_ "kill -$sig" test $rc -eq 99
+ else
+ # The exit trap is not installed, so that the shell should exit
+ # with status 128+n when receiving signal number n. But don't
+ # be too strict in the check, as POSIX only says that "The exit
+ # status of a command that terminated because it received a
+ # signal shall be reported as greater than 128".
+ command_ok_ "kill -$sig" test $rc -gt 128
+ fi
+ unset rc
+done
+
+: Non-existent program.
+# Solaris 10 /bin/sh erroneously exit with success right away when the
+# following three conditions are met at the same time:
+# 1. the `errexit' flag is active,
+# 2. an exit trap is installed, and
+# 3. a non-existing command is issued.
+# Note that the non-existent command is issued as the last command to
+# the shell in the next line; this is deliberate.
+if $SHELL -c 'set -e; trap "exit \$?" 0; non-existent-program'; then
+ maybe_todo=TODO reason="known Solaris /bin/sh bug"
+else
+ maybe_todo="" reason=""
+fi
+$SHELL -c "$init; non-existent-program; :"
+command_ok_ "command not found" -D "$maybe_todo" -r "$reason" \
+ -- test $? -gt 0
+
+: Non-executable command.
+test -f Makefile && test ! -x Makefile || \
+ framowork_failure_ "no proper Makefile in the current directory"
+$SHELL -c "$init; ./Makefile; :"
+command_ok_ "permission denied" test $? -gt 0
+
+: Syntax errors in the test code.
+$SHELL -c "$init; if :; then"
+command_ok_ "syntax error 1" test $? -gt 0
+$SHELL -c "$init; fi"
+command_ok_ "syntax error 2" test $? -gt 0
+
+:
+++ /dev/null
-#! /bin/sh
-# Copyright (C) 2011 Free Software Foundation, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2, or (at your option)
-# any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program. If not, see <http://www.gnu.org/licenses/>.
-
-# Sanity check for the automake testsuite.
-# Check that. in case of failing commands, the correct exit status is
-# passed to the exit trap installed by the `./defs' script.
-# Also check that the `errexit' shell flag is active.
-
-. ./defs-static || exit 99
-
-for st in 1 2 3 4 5 77 99 126 127 128 129 130 255; do
-
- echo "* Try: Exit $st"
- $SHELL -c ". ./defs; Exit $st; :"
- rc=$?
- echo "* rc=$rc"
- echo
- test $rc -eq $st || exit 1
-
- echo "* Try: sh -c 'exit $st'"
- $SHELL -c ". ./defs; sh -c 'exit $st'; :"
- rc=$?
- echo "* rc=$rc"
- echo
- test $rc -eq $st || exit 1
-
-done
-
-# Disabled: some lesser shells such as Solaris /bin/sh might fail
-# on this.
-#echo "* Try: non-existent-program"
-#$SHELL -c ". ./defs; non-existent-program; :"
-#rc=$?
-#echo "* rc=$rc"
-#echo
-#test $rc -eq 127 || exit 1
-
-for sig in 1 2 13 15; do
-
- echo "* Try: kill -$sig \$\$"
- if test $sig -eq 2; then
- # Some Korn shells might otherwise get a spurious SIGINT
- # signal when one is sent to the child $SHELL.
- trap : 2
- fi
- $SHELL -c ". ./defs; kill -$sig \$\$; :"
- rc=$?
- if test $sig -eq 2; then
- # Reset default SIGINT handler as portably as possible.
- trap 2 || trap - 2
- fi
- echo "* rc=$rc"
- echo
- if test x"$sh_errexit_works" = x"yes"; then
- # The exit trap should turn into an hard errors any failure
- # caused by signals.
- test $rc -eq 99 || exit 1
- else
- # The exit trap is not installed, so that the shell should exit
- # with status 128+n when receiving signal number n. But don't
- # be too strict in the check, as POSIX only says that "The exit
- # status of a command that terminated because it received a
- # signal shall be reported as greater than 128".
- test $rc -gt 128 || exit 1
- fi
-
-done
-
-:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Sanity check for the automake testsuite.
+# Check creation/removal of temporary test working directory by `./defs'.
+
+. ./defs || Exit 1
+
+test x"$sh_errexit_works" = x"yes" || skip_ "no working shell exit trap"
+
+# We still need a little hack to make ./defs work outside automake's
+# tree `tests' subdirectory. Not a big deal.
+sed "s|^testbuilddir=.*|testbuilddir='`pwd`'|" ../defs-static >defs-static
+diff ../defs-static defs-static && Exit 99
+cp ../defs .
+
+set +e
+
+unset am_explicit_skips stderr_fileno_
+AM_TESTS_REEXEC=no; export AM_TESTS_REEXEC
+
+$SHELL -c '. ./defs; (exit 77); exit 77' dummy.test
+test $? -eq 77 || Exit 1
+
+am_explicit_skips=no $SHELL -c '. ./defs; sh -c "exit 77"' dummy.test
+test $? -eq 77 || Exit 1
+
+am_explicit_skips=yes $SHELL -c '. ./defs; (exit 77); exit 77' dummy.test
+test $? -eq 78 || Exit 1
+
+am_explicit_skips=y $SHELL -c '. ./defs; sh -c "exit 77"' dummy.test
+test $? -eq 78 || Exit 1
+
+am_explicit_skips=yes $SHELL -c '. ./defs; Exit 77' dummy.test
+test $? -eq 77 || Exit 1
+
+am_explicit_skips=y $SHELL -c '. ./defs; skip_ "foo"' dummy.test
+test $? -eq 77 || Exit 1
+
+:
: > b
: > c
-stat a b c || : # for debugging
+stat a b c || : # For debugging.
is_newest c a
is_newest b a
touch -r c d
-stat c d || : # for debugging
+stat c d || : # For debugging.
is_newest c d
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Sanity check for the automake testsuite.
+# Make sure that $me gets automatically defined by `./defs', and that it
+# can be overridden by the test script.
+
+am_create_testdir=no
+. ./defs || Exit 1
+
+plan_ 9
+
+AM_TESTS_REEXEC=no; export AM_TESTS_REEXEC
+
+# This test becomes cumbersome if we keep the `errexit' shell flag set.
+# And removing it is no big deal, as this test is a TAP-based one, so
+# that false positives remain very unlikely.
+set +e
+
+do_check ()
+{
+ $SHELL -c '. ./defs && echo me=$me' "$1" | grep "^me=$2$"
+ command_ok_ "me=$1" test $? -eq 0
+}
+
+do_check foo-bar-.test 'foo-bar-'
+do_check _foo__bar.test '_foo__bar'
+do_check 012.test '012'
+do_check foo.bar 'foo\.bar'
+do_check a.b.c.test 'a\.b\.c'
+
+# If we override $me, ./defs should not modify it.
+
+s=`$SHELL -c 'me=foo.test && . ./defs && echo me=$me' bad.test`
+command_ok_ "override of \$me before ./defs causes no error" \
+ test $? -eq 0
+
+r='ok'
+printf '%s\n' "$s" | grep '^me=foo\.test$' || r='not ok'
+printf '%s\n' "$s" | grep 'me=bad' && r='not ok'
+result_ "$r" "override of \$me before ./defs is honored"
+unset r
+
+# Overriding $me after sourcing ./defs-static should work.
+s=`$SHELL -c '. ./defs-static && me=zardoz &&
+ . ./defs && echo me=$me' bad.test`
+command_ok_ "override of \$me after ./defs-static causes no error" \
+ test $? -eq 0
+
+r='ok'
+printf '%s\n' "$s" | grep '^me=zardoz$' || r='not ok'
+printf '%s\n' "$s" | grep 'me=bad' && r='not ok'
+result_ "$r" "override of \$me after ./defs-static is honored"
+unset r
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Sanity check for the automake testsuite.
+# Check that automatic re-execution of test script with the
+# configure-time $SHELL.
+
+required=bash
+. ./defs || Exit 1
+
+unset AM_TESTS_REEXEC BASH_VERSION || :
+
+# This is not optimal, but it's much better than writing wrapper
+# scripts acting as "fake" shells.
+
+bash -c 'test -n "$BASH_VERSION"' || skip_ "bash shell not found"
+
+for shell in /bin/sh /bin/ksh sh ksh dash pdksh __no_shell__; do
+ test $shell = __no_shell__ && skip_ "can't find any non-bash shell"
+ $shell -c 'test -n "$BASH_VERSION"' && continue
+ break
+done
+echo "shell='$shell'"
+
+cwd=`pwd` || Exit 99
+
+cp ../defs .
+
+#
+# Check how to default, force or prevent a re-execution.
+#
+
+cat > run-with-bash.test <<'END'
+#!/bin/false
+. ./defs
+# Ensure that the script gets re-executed with bash. Also ensure that
+# non-standard syntax used after the inclusion of `./defs' doesn't cause
+# non-bash shells to fail.
+# Subshell required to prevent some shells (e.g., Solaris 10 /bin/sh)
+# from only complaining on stderr but then exiting with exit status 0.
+(foo=abac && test xbxc = ${foo//a/x} && test -n "$BASH_VERSION")
+END
+
+sed -e "s|^testbuilddir=.*|testbuilddir='$cwd'|" \
+ -e 's|^SHELL=.*$|SHELL=bash; export SHELL|' \
+ < ../defs-static >defs-static
+
+$shell -x run-with-bash.test
+
+AM_TESTS_REEXEC='' $shell run-with-bash.test
+AM_TESTS_REEXEC=yes $shell run-with-bash.test
+AM_TESTS_REEXEC=y $shell run-with-bash.test
+AM_TESTS_REEXEC=true $shell run-with-bash.test
+AM_TESTS_REEXEC=1 $shell run-with-bash.test
+AM_TESTS_REEXEC=no $shell run-with-bash.test && Exit 1
+AM_TESTS_REEXEC=n $shell run-with-bash.test && Exit 1
+AM_TESTS_REEXEC=false $shell run-with-bash.test && Exit 1
+AM_TESTS_REEXEC=0 $shell run-with-bash.test && Exit 1
+
+#
+# Check message about the re-execution.
+#
+
+cat > dummy.test <<'END'
+#!/bin/sh
+. ./defs
+:
+END
+chmod a+x dummy.test
+
+mkdir sub
+cp dummy.test defs sub
+sed -e "s|^testbuilddir=.*|testbuilddir='$cwd'|" \
+ < ../defs-static > defs-static
+sed -e "s|^testbuilddir=.*|testbuilddir='$cwd/sub'|" \
+ < ../defs-static > sub/defs-static
+
+./dummy.test a b | grep "^dummy: exec $SHELL \\./dummy\\.test a b$"
+
+for am_sh in $shell bash; do
+ $am_sh dummy.test a b c \
+ | grep "^dummy: exec $SHELL dummy\\.test a b c$"
+ $am_sh ./dummy.test a b c \
+ | grep "^dummy: exec $SHELL \\./dummy\\.test a b c$"
+ cd sub
+ $am_sh ../dummy.test a b \
+ | grep "dummy: exec $SHELL \\.\\./dummy\\.test a b$"
+ cd ..
+ $am_sh "$cwd/dummy.test" a -b c- \
+ | grep "^dummy: exec $SHELL $cwd/dummy\\.test a -b c-$"
+ $am_sh sub/dummy.test 1 2 3 4 \
+ | grep "^dummy: exec $SHELL sub/dummy\\.test 1 2 3 4$"
+done
+
+#
+# Check that arguments passed to a test script are preserved by a re-exec.
+#
+
+cat > checkargs.test <<'END'
+. ./defs
+test $# -eq 3 && test x"$1" = x'a' && test x"$2" = x && test x"$3" = x"-e"
+END
+
+$SHELL checkargs.test a '' -e
+$SHELL ./checkargs.test a '' -e
+$SHELL "$cwd/checkargs.test" a '' -e
+cd sub
+$SHELL ../checkargs.test a '' -e
+cd ..
+cp checkargs.test sub
+$SHELL sub/checkargs.test a '' -e
+
+:
# Test subroutines to report warnings, and to signal failures, skips
# and hard errors.
+unset stderr_fileno_ || :
+
. ./defs || Exit 1
set +e
(fail_ foo) 2>&1 1>&5 | grep "^$me: failed test: foo" || Exit 1
(skip_ foo); test $? -eq 77 || Exit 1
(skip_ foo) 2>&1 1>&5 | grep "^$me: skipped test: foo" || Exit 1
+(fatal_ foo); test $? -eq 99 || Exit 1
+(fatal_ foo) 2>&1 1>&5 | grep "^$me: hard error: foo" || Exit 1
(framework_failure_ foo); test $? -eq 99 || Exit 1
(framework_failure_ foo) 2>&1 1>&5 \
| grep "^$me: set-up failure: foo" || Exit 1
(fail_ foo) 6>&1 1>&5 | grep "^$me: failed test: foo" || Exit 1
(skip_ foo); test $? -eq 77 || Exit 1
(skip_ foo) 6>&1 1>&5 | grep "^$me: skipped test: foo" || Exit 1
+(fatal_ foo); test $? -eq 99 || Exit 1
+(fatal_ foo) 6>&1 1>&5 | grep "^$me: hard error: foo" || Exit 1
(framework_failure_ foo); test $? -eq 99 || Exit 1
(framework_failure_ foo) 6>&1 1>&5 \
| grep "^$me: set-up failure: foo" || Exit 1
sed 's/^/ | /' stderr >&2
}
+AM_TESTS_REEXEC=no; export AM_TESTS_REEXEC
+
if $SHELL -c '. ../defs' dummy.test 2>stderr; then
show_stderr
Exit 1
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Sanity check for the automake testsuite.
+# Check the `seq_' subroutine.
+
+. ./defs || Exit 1
+
+unset stderr_fileno_ || :
+
+: One-argument form.
+exp="\
+1
+2
+3
+4
+5"
+got=`seq_ 5` || Exit 1
+test x"$exp" = x"$got" || Exit 1
+
+: Two-arguments form.
+exp="\
+7
+8
+9
+10
+11"
+got=`seq_ 7 11` || Exit 1
+test x"$exp" = x"$got" || Exit 1
+
+: Three-arguments form [1].
+exp="\
+120
+125
+130
+135"
+got=`seq_ 120 5 135` || Exit 1
+test x"$exp" = x"$got" || Exit 1
+
+: Three-arguments form [2].
+exp="\
+13
+17
+21"
+got=`seq_ 13 4 23` || Exit 1
+test x"$exp" = x"$got" || Exit 1
+
+: No argument is an error.
+st=0
+(seq_) 2>stderr || st=$?
+test $st -eq 99
+grep 'seq_: missing argument' stderr
+
+: Four or more arguments is an error.
+for args in '1 1 2 1' '1 1 1 1 1 1'; do
+ st=0
+ (seq_ $args) 2>stderr || st=$?
+ test $st -eq 99
+ grep 'seq_: too many arguments' stderr
+done
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Sanity check for the automake testsuite.
+# Make sure that $using_tap gets automatically defined by `./defs-static',
+# but can be overridden by the individual tests.
+
+. ./defs-static || exit 1
+
+set -ex
+
+$SHELL -c '. ./defs-static && test $using_tap = yes' foo.tap
+$SHELL -c '. ./defs-static && test $using_tap = no' foo.test
+$SHELL -c '. ./defs-static && test $using_tap = no' tap
+$SHELL -c '. ./defs-static && test $using_tap = no' tap.test
+$SHELL -c '. ./defs-static && test $using_tap = no' foo-tap
+
+$SHELL -c 'using_tap=no; . ./defs-static; test $using_tap = no' foo.tap
+$SHELL -c 'using_tap=yes; . ./defs-static; test $using_tap = yes' foo.test
+
+:
#------------------------------------------------------------------
-: Leading spaces
+: Leading spaces.
cat > input <<END
1
#------------------------------------------------------------------
-: Leading tab
+: Leading tab.
cat > input <<END
${tab}1
#------------------------------------------------------------------
-: No leading whitespace
+: No leading whitespace.
cat > input <<END
1
#------------------------------------------------------------------
-: Leading empty lines are ignored [1]
+: Leading empty lines are ignored [1].
cat > input <<END
#------------------------------------------------------------------
-: Leading empty lines are ignored [2]
+: Leading empty lines are ignored [2].
cat > input <<END
#------------------------------------------------------------------
-: More elaborated parallel use
+: More elaborated parallel use.
cat > input <<END
x
. ./defs || Exit 1
-set -e
-
cat >> configure.in <<'EOF'
AM_SILENT_RULES
AC_OUTPUT
"END" return EOF;
.
%%
+/* Avoid possible link errors. */
+int yywrap (void)
+{
+ return 1;
+}
+int main (void)
+{
+ return 0;
+}
EOF
cp foo.l sub/bar.l
# Check silent-rules mode for Lex.
# Keep this in sync with sister test `silent-lex-gcc.test'.
-required='flex'
+required='cc flex'
. ./defs || Exit 1
mkdir sub
"END" return EOF;
.
%%
+/* Avoid possible link errors. */
+int yywrap (void)
+{
+ return 1;
+}
+int main (void)
+{
+ return 0;
+}
EOF
cp foo.l sub/bar.l
"END" return EOF;
.
%%
+/* Avoid possible link errors. */
+int yywrap (void)
+{
+ return 1;
+}
EOF
cat > foo6.y <<'EOF'
%{
$AUTOMAKE --add-missing
$AUTOCONF
+# Sanity check: make sure the cache variables we force are really
+# used by configure.
+$FGREP am_cv_CC_dependencies_compiler_type configure
+$FGREP am_cv_CXX_dependencies_compiler_type configure
+
# Force gcc ("fast") depmode.
+depmodes="am_cv_CC_dependencies_compiler_type=gcc \
+ am_cv_CXX_dependencies_compiler_type=gcc"
# This apparently useless "for" loop is here to simplify the syncing
-# with sister test `silent-many-gcc.test'.
+# with sister test `silent-many-generic.test'.
for config_args in \
- am_cv_CC_dependencies_compiler_type=gcc
+ "$depmodes"
do
./configure $config_args --enable-silent-rules
# sister test `silent-many-gcc.test', which requires the GNU compilers
# and forces the use of gcc depmode.
-# FIXME: generic C++/Fortran compilers should suffice here
-required='g++ gfortran flex yacc'
+required='cc c++ fortran fortran77 flex yacc'
. ./defs || Exit 1
# Avoids too much code duplication.
cat >>configure.in <<'EOF'
AM_SILENT_RULES
AM_PROG_CC_C_O
-AC_PROG_CXX
AC_PROG_F77
AC_PROG_FC
AC_PROG_LEX
AC_PROG_YACC
+AC_PROG_CXX
+
+# The SunStudio C++ compiler is unfortunately named `CC' (yuck!),
+# and this can cause problems with our grepping checks on the
+# output from make. Avoid these problems by invoking a wrapper
+# script, as filtering the make output proved too fragile.
+case " $CXX " in
+ *' CC '*|*'/CC '*)
+ AC_MSG_WARN([the C++ compiler '$CXX' seems to be named 'CC'])
+ AC_MSG_WARN([it will be wrapped with the custom script 'am--cxx'])
+ echo '#!/bin/sh' > bin/am--cxx
+ echo 'PATH=$saved_PATH; export PATH' >> bin/am--cxx
+ echo "exec $CXX \${1+"\$@"}" >> bin/am--cxx
+ chmod a+x bin/am--cxx
+ CXX=am--cxx
+esac
+
AC_CONFIG_FILES([sub/Makefile])
AC_OUTPUT
EOF
"END" return EOF;
.
%%
+/* Avoid possible link errors. */
+int yywrap (void)
+{
+ return 1;
+}
EOF
cat > foo6.y <<'EOF'
%{
cp foo5.l sub/baz5.l
cp foo6.y sub/baz6.y
+mkdir bin
+saved_PATH=$PATH; export saved_PATH
+PATH=`pwd`/bin$PATH_SEPARATOR$PATH; export PATH
+
$ACLOCAL
$AUTOMAKE --add-missing
$AUTOCONF
# Check silent-rules mode for Yacc.
# Keep this in sync with sister test `silent-yacc-gcc.test'.
-required=yacc
+required='cc yacc'
. ./defs || Exit 1
mkdir sub
# Please keep this file in sync with silent2.test.
+required=cc
. ./defs || Exit 1
mkdir sub
# Please keep this file in sync with silent4.test and silent9.test.
-required=libtoolize
+required='cc libtoolize'
. ./defs || Exit 1
mkdir sub
AUTOMAKE_fails
$AUTOMAKE -Wno-error
-# AM_SILENT_RULES should turn off the warning
+# AM_SILENT_RULES should turn off the warning.
$sleep
echo 'AM_SILENT_RULES' >> configure.in
$ACLOCAL
./configure --disable-silent-rules
-# Make sure that all labels work in silent-mode
+# Make sure that all labels work in silent-mode.
$MAKE V=0 dvi html info ps pdf >stdout || { cat stdout; Exit 1; }
cat stdout
grep 'DVIPS foo.ps' stdout || Exit 1
# Please keep this file in sync with silent3.test and silent4.test.
-required="libtoolize"
+required='cc libtoolize'
. ./defs || Exit 1
mkdir sub
--- /dev/null
+#!/bin/sh
+# Copyright (C) 2010, 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check silent-rules mode for C++.
+# This test requires the GNU C++ compiler; keep it in sync with sister
+# test `silentcxx.test', which should work with generic compilers.
+
+required=g++
+. ./defs || Exit 1
+
+mkdir sub
+
+cat >>configure.in <<'EOF'
+AM_SILENT_RULES
+AC_PROG_CXX
+AC_CONFIG_FILES([sub/Makefile])
+AC_OUTPUT
+EOF
+
+cat > Makefile.am <<'EOF'
+# Need generic and non-generic rules.
+bin_PROGRAMS = foo1 foo2
+foo1_SOURCES = foo.cpp baz.cxx quux.cc
+foo2_SOURCES = $(foo1_SOURCES)
+foo2_CXXFLAGS = $(AM_CXXFLAGS)
+SUBDIRS = sub
+EOF
+
+cat > sub/Makefile.am <<'EOF'
+AUTOMAKE_OPTIONS = subdir-objects
+# Need generic and non-generic rules.
+bin_PROGRAMS = bar1 bar2
+bar1_SOURCES = bar.cpp
+bar2_SOURCES = $(bar1_SOURCES)
+bar2_CXXFLAGS = $(AM_CXXFLAGS)
+EOF
+
+cat > foo.cpp <<'EOF'
+using namespace std; /* C compilers fail on this. */
+int main() { return 0; }
+EOF
+
+# Let's try out other extensions too.
+echo 'class Baz { public: int i; };' > baz.cxx
+echo 'class Quux { public: bool b; };' > quux.cc
+
+cp foo.cpp sub/bar.cpp
+
+$ACLOCAL
+$AUTOMAKE --add-missing
+$AUTOCONF
+
+# Sanity check: make sure the cache variable we force is really used
+# by configure.
+$FGREP am_cv_CXX_dependencies_compiler_type configure
+
+# Force gcc ("fast") depmode.
+# This apparently useless "for" loop is here to simplify the syncing
+# with sister test `silentcxx.test'.
+for config_args in \
+ am_cv_CXX_dependencies_compiler_type=gcc
+do
+ ./configure $config_args --enable-silent-rules
+ $MAKE >stdout || { cat stdout; Exit 1; }
+ cat stdout
+
+ $EGREP ' (-c|-o)' stdout && Exit 1
+ grep 'mv ' stdout && Exit 1
+
+ grep 'CXX .*foo\.' stdout
+ grep 'CXX .*baz\.' stdout
+ grep 'CXX .*quux\.' stdout
+ grep 'CXX .*bar\.' stdout
+ grep 'CXXLD .*foo1' stdout
+ grep 'CXXLD .*bar1' stdout
+ grep 'CXXLD .*foo2' stdout
+ grep 'CXXLD .*bar2' stdout
+
+ # Ensure a clean rebuild.
+ $MAKE clean
+
+ $MAKE V=1 >stdout || { cat stdout; Exit 1; }
+ cat stdout
+
+ grep ' -c ' stdout
+ grep ' -o ' stdout
+
+ $EGREP '(CC|CXX|LD) ' stdout && Exit 1
+
+ # Ensure a clean reconfiguration/rebuild.
+ $MAKE clean
+ $MAKE maintainer-clean
+
+done
+
+:
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Check silent-rules mode for C++.
+# This test should work with generic C++ compilers; keep it in sync with
+# sister test `silentcxx-gcc.test', which requires the GNU C++ compiler
+# and forces the use of gcc depmode.
-required='g++' # FIXME: any decent C++ compiler should be OK
+required=c++
. ./defs || Exit 1
mkdir sub
EOF
cat > foo.cpp <<'EOF'
-using namespace std; /* C compilers fail on this */
-int main() { return 0; }
+using namespace std; /* C compilers fail on this. */
+int main (void) { return 0; }
EOF
-# let's try out other extensions too
+# Let's try out other extensions too.
echo 'class Baz { public: int i; };' > baz.cxx
echo 'class Quux { public: bool b; };' > quux.cc
$AUTOMAKE --add-missing
$AUTOCONF
-# configure once for fastdep, once for non-fastdep, once for nodep
+# Sanity check: make sure the cache variable we force is really used
+# by configure.
+$FGREP am_cv_CXX_dependencies_compiler_type configure
+
+# Force dependency tracking explicitly, so that slow dependency
+# extractors are not rejected. Try also with dependency tracking
+# explicitly disabled.
for config_args in \
- '' \
- am_cv_CC_dependencies_compiler_type=gcc \
- --disable-dependency-tracking
+ --enable-dependency-tracking --disable-dependency-tracking
do
./configure $config_args --enable-silent-rules
$MAKE >stdout || { cat stdout; Exit 1; }
grep ' -c ' stdout
grep ' -o ' stdout
- $EGREP '(CC|CXX|LD) ' stdout && Exit 1
+ $EGREP '(CXX|LD) ' stdout && Exit 1
# Ensure a clean reconfiguration/rebuild.
$MAKE clean
# Check silent-rules mode for Fortran 77.
# Keep this ins sync with the sister test silentf90.test.
-required='gfortran' # FIXME: any working Fortran compiler should be OK!
+required=fortran77
. ./defs || Exit 1
mkdir sub
# Check silent-rules mode for Fortran 90.
# Keep this ins sync with the sister test silentf77.test.
-required='gfortran' # FIXME: any working Fortran compiler should be OK!
+required=fortran
. ./defs || Exit 1
mkdir sub
# AM_DEFAULT_SOURCE_EXT
-required=g++
+required='cc c++'
. ./defs || Exit 1
cat >> configure.in << 'END'
# Regression test for multiple rules being generated for each target when
# conditionals are present.
-# From Richard Boulton
+# From Richard Boulton.
. ./defs || Exit 1
# The true/false example from the manual, plus a check for _SHORTNAME.
-required=gcc
+required='cc native'
. ./defs || Exit 1
cat >> configure.in << 'END'
./false | grep false
test -f ./t-false.o
test -f ./f-false.o
+
+:
# with one extra indirection in the sources (PR/315), and
# use of _CPPFLAGS (PR/337).
-required=gcc
+required='cc native'
. ./defs || Exit 1
cat >> configure.in << 'END'
# Test to make sure the standard include order is stable.
# Report by Kent Boortz.
+required=cc
. ./defs || Exit 1
cat >> configure.in << 'END'
# Test for install-strip.
+required=cc
. ./defs || Exit 1
cat >> configure.in << 'END'
# Ensure install-strip works when STRIP consists of more than one word.
# This test needs GNU binutils strip. See sister test strip3.
-required=strip
+required='cc strip'
. ./defs || Exit 1
cat >> configure.in << 'END'
# Ensure install-strip works when STRIP consists of more than one word.
# This test needs GNU binutils strip. Libtool variant.
-required='libtoolize strip'
+required='cc libtoolize strip'
. ./defs || Exit 1
cat >> configure.in << 'END'
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Test to make sure that adding a new directory works.
-# This test runs `make' from the top-level directory, subdir8.test
-# do it from a subdirectory.
+# This test runs `make' from the top-level directory, the sister test
+# `subdir8.test' do it from a subdirectory.
# PR automake/46
-# This test assumes that the `make' utility is able to start
-# over and reload Makefiles which have been remade (a non-POSIX feature).
-# See also the related test subdir8.test.
-required='GNUmake gcc'
. ./defs || Exit 1
cat >> configure.in << 'END'
-AC_PROG_CC
-AM_PROG_CC_C_O
AC_OUTPUT
END
-cat > Makefile.am << 'END'
-bin_PROGRAMS = wish
-wish_SOURCES = a.c
-END
-
-cat > a.c << 'END'
-#include <stdio.h>
-int main ()
-{
- printf ("hi liver!\n");
- return 0;
-}
-END
+: > Makefile.am
$ACLOCAL
$AUTOCONF
-$AUTOMAKE --include-deps --copy --add-missing
+$AUTOMAKE
./configure
$MAKE
sed <configure.in >configure.tmp -e '/^AC_OUTPUT$/i\
AC_CONFIG_FILES([maude/Makefile])\
m4_include([confile.m4])\
-' # last newline required by older OpenBSD sed
+' # Last newline required by older OpenBSD sed.
mv -f configure.tmp configure.in
-cat configure.in # might be useful for debugging
+cat configure.in # For debugging.
: > confile.m4
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Test to make sure that adding a new directory works, even from
-# subdirectories. (subdir5.test makes sure it works when make
-# is run from the top-level directory.)
+# subdirectories. The sister test `subdir5.test' makes sure it works
+# when make is run from the top-level directory.
# PR automake/46
-# This test assumes that the `make' utility is able to start
-# over and reload Makefiles which have been remade (a non-POSIX feature).
-# See also the related test subdir5.test.
-required='GNUmake gcc'
. ./defs || Exit 1
cat >> configure.in << 'END'
-AC_PROG_CC
-AM_PROG_CC_C_O
m4_include([confiles.m4])
MORE_DEFS
AC_OUTPUT
mkdir sub
-cat > sub/Makefile.am << 'END'
-bin_PROGRAMS = wish
-wish_SOURCES = a.c
-END
-
-cat > sub/a.c << 'END'
-#include <stdio.h>
-int main ()
-{
- printf ("hi liver!\n");
- return 0;
-}
-END
+: > sub/Makefile.am
mkdir m4
echo 'AC_DEFUN([MORE_DEFS], [])' > m4/moredefs.m4
$ACLOCAL -I m4
$AUTOCONF
-$AUTOMAKE --copy --add-missing
+$AUTOMAKE
./configure
$MAKE
# A bug occurred where subdirs do not have all-recursive or
# all-recursive-am which depended on BUILT_SOURCES.
+required=cc
. ./defs || Exit 1
mkdir lib
# PR 492: Test asm subdir-objects.
-required=gcc # avoid compiler errors.
+required=gcc
. ./defs || Exit 1
cat > configure.in << END
# See also sister "grepping" test `subobj11b.test', and related test
# `subobj11c.test'.
+required=cc
. ./defs || Exit 1
cat >> configure.in << 'END'
cat > Makefile.am << 'END'
AUTOMAKE_OPTIONS = subdir-objects
bin_PROGRAMS = foo
-## the `.//' is meant
+## The `.//' below is meant.
foo_SOURCES = .//src/foo.c
END
# Automatic dependency tracking with subdir-objects option active:
# check for a pathological case of slash-collapsing in the name of
# included makefile fragments (containing dependency info).
-# See also related tests `subobj11a.test' and `subobj11b.test'
+# See also related tests `subobj11a.test' and `subobj11b.test'.
. ./defs || Exit 1
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Test of subdir make dist rules.
-# From Robert Collins
+# From Robert Collins.
+required=cc
. ./defs || Exit 1
cat >> configure.in << 'END'
END
cat > Makefile.am << 'END'
+AUTOMAKE_OPTIONS = subdir-objects
bin_PROGRAMS = wish
wish_SOURCES = generic/a.c
+
+.PHONY: test-distdir test-build
+test-distdir: distdir
+ ls -l $(distdir) $(distdir)/* ;: For debugging.
+ test ! -r $(distdir)/a.c
+ test -f $(distdir)/generic/a.c
+test-build: all
+ ls -l . generic ;: For debugging.
+ test -f generic/a.$(OBJEXT)
+ test -f wish$(EXEEXT)
+ test ! -r a.$(OBJEXT)
+ test ! -r a$(EXEEXT)
+ test ! -r generic/wish$(EXEEXT)
+ test ! -r generic/a$(EXEEXT)
END
mkdir generic
-: > generic/a.c
: > generic/Makefile.am
+cat > generic/a.c << 'END'
+int main (void)
+{
+ return 0;
+}
+END
+
$ACLOCAL
$AUTOCONF
$AUTOMAKE --include-deps --copy --add-missing
./configure
+$MAKE test-distdir
$MAKE dist
+$MAKE test-build
:
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Test of subdir make distclean rules.
-# From Robert Collins
+# From Robert Collins.
-required=gcc
+required=cc
. ./defs || Exit 1
cat >> configure.in << 'END'
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Test for PR 312.
+#
+# == Report ==
+# When using non-recursive make to build a libtoolize-library from
+# sources in a subdirectory, 'make distcheck' fails because of incomplete
+# cleanup. "make clean" tries to remove `*.o' and `.../<file>.lo' but
+# forgets `.../<file>.o'.
-required='libtoolize g++'
+required='c++ libtoolize'
. ./defs || Exit 1
cat > configure.in << END
cat > Makefile.am << 'END'
noinst_LTLIBRARIES = libfoo.la
-libfoo_la_SOURCES = src/foo.cc .//src/bar.cc # the `.//' is meant.
+libfoo_la_SOURCES = src/foo.cc .//src/bar.cc # The `.//' is meant.
.PHONY: print
print:
@echo BEG1: "$(LTCXXCOMPILE)" :1END
$AUTOCONF
$AUTOMAKE -a
-# Skip this test on configure errors (e.g., broken C++ compilers).
-./configure || Exit 77
+./configure
# Ensure './libtool --help' will use the right tool versions.
export AUTOCONF AUTOMAKE
# Check subpackage handling.
-required='gcc yacc'
+required='cc yacc'
. ./defs || Exit 1
mkdir m4
EOF
cat >LDADD.c <<'EOF'
-int lib ();
+int lib (void);
-int
-main ()
+int main (void)
{
return lib ();
}
cat >lib/src/x.c <<'EOF'
#include <config.h>
-int lib ()
+int lib (void)
{
return 0;
}
./configure
$MAKE
$MAKE distcheck
-test ! -d subpack-1 # make sure distcheck cleans up after itself
+test ! -d subpack-1 # Make sure distcheck cleans up after itself.
test -f subpack-1.tar.gz
+
+:
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# make sure different subpackages may share files and directories.
+# Make sure different subpackages may share files and directories.
. ./defs || Exit 1
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2003, 2006, 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# If the last line of a automake-rewritten definition is made only of
+# @substitutions@, automake should take care of appending an empty
+# variable to make sure that line cannot end up substituted as a blank
+# line (that would confuse HP-UX Make).
+# These checks have been introduced in commit `Release-1-9-254-g9d0eaef'
+# into the former test `subst2.test'.
+
+. ./defs || Exit 1
+
+# These are deliberately quite long, so that the xxx_PROGRAMS definition
+# in Makefile.am below will be split on multiple lines, with the last
+# line containing only @substituted@ stuff that expands to empty (this is
+# required to expose the bug we are testing).
+v1=ABCDEFGHIJKLMNOPQRSTUVWX
+v2=ABCDEFGHIJKLMNOPQRSTUVWXY
+v3=ABCDEFGHIJKLMNOPQRSTUVWXYZ
+
+# Literal backslash for use by grep.
+bs='\\'
+
+cat >> configure.in <<END
+AC_SUBST([A], [''])
+AC_SUBST([$v1], [''])
+AC_SUBST([$v2], [''])
+AC_SUBST([$v3], [''])
+AC_OUTPUT
+END
+
+cat >Makefile.am <<END
+AUTOMAKE_OPTIONS = no-dependencies
+CC = false
+EXEEXT =
+
+## The "x" and "zardoz" strings and the use of '+=' are there to ensure
+## that these variables get rewritten by Automake.
+noinst_PROGRAMS = x @$v1@ @$v2@ @$v3@
+bin_PROGRAMS = @A@
+bin_PROGRAMS += @$v1@ @$v2@ @$v3@
+check_PROGRAMS = zardoz \$(noinst_PROGRAMS)
+
+## Required whenever there are @substituted@ values in the
+## PROGRAMS primary, otherwise automake will complain.
+EXTRA_PROGRAMS =
+
+print-programs:
+ @echo BEG1: \$(noinst_PROGRAMS) :END1
+ @echo BEG2: \$(bin_PROGRAMS) :END2
+ @echo BEG3: \$(check_PROGRAMS) :END3
+END
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE
+# For debugging.
+$EGREP -n 'ABCD|am__empty' Makefile.in
+# Sanity check.
+test `$EGREP -c "^[ $tab]*@$v2@ @$v3@[ $tab]*$bs?$" Makefile.in` -eq 3
+
+./configure
+{
+ sed -n '/^noinst_PROGRAMS *=/,/[^\\]$/p' Makefile
+ sed -n '/^bin_PROGRAMS *=/,/[^\\]$/p' Makefile
+ sed -n '/^check_PROGRAMS *=/,/[^\\]$/p' Makefile
+ sed -n '/^am__EXEEXT.*=/,/[^\\]$/p' Makefile
+} >t-programs
+cat t-programs
+grep '^ *$' t-programs && Exit 1
+
+$MAKE print-programs >stdout || { cat stdout; Exit 1; }
+cat stdout
+grep '^BEG1: x :END1$' stdout
+grep '^BEG2: :END2$' stdout
+grep '^BEG3: zardoz x :END3$' stdout
+
+am__empty=X $MAKE -e print-programs >stdout || { cat stdout; Exit 1; }
+cat stdout
+grep '^BEG1: x X :END1$' stdout
+grep '^BEG2: X :END2$' stdout
+grep '^BEG3: zardoz x X :END3$' stdout
+
+:
# Test installation with substitutions. This test is based on nobase.test.
-required='gcc'
+required=cc
. ./defs || Exit 1
cat >> configure.in <<'EOF'
$MAKE
$MAKE test-install-nothing-data
$MAKE test-install-nothing-exec
+
+:
# Test for bug in variable substitution references, where
# undefined variables break later substitutions.
-required=gcc
+required=cc
. ./defs || Exit 1
cat >> configure.in << 'END'
# Test for bug in variable substitution references when left hand
# pattern is null.
-# Report from Richard Boulton
+# Report from Richard Boulton.
-required=gcc
+required=cc
. ./defs || Exit 1
cat >> configure.in << 'END'
END
cat > Makefile.am << 'END'
-
var1 = dlmain
-
var2 = $(var1:=.)
-
helldl_SOURCES = $(var2:=c)
-
bin_PROGRAMS = helldl
-
got:
@echo $(helldl_SOURCES) $(helldl_OBJECTS) >got
-
END
-# Ignore user CFLAGS.
-CFLAGS=
-export CFLAGS
-
$ACLOCAL
$AUTOCONF
$AUTOMAKE -a
libfake@SUBST@_a_SOURCES = abra.c kadabra.c
-# then we override the target rule:
+# Then we override the target rule:
libfake@SUBST@.a: Makefile $(libfake@SUBST@_a_OBJECTS) $(libfake@SUBST@_a_DEPENDENCIES)
@echo here we do some custom stuff, instead of invoking the linker
END
# some make implementations to automatically chain suffix rules.
# See automake bug#7824 and bug#7670.
+required=cc
. ./defs || Exit 1
+plan_ 8
+
cat >> configure.in <<'END'
AC_PROG_CC
AC_OUTPUT
echo 'return 0;' > foo.c1
-$ACLOCAL
-$AUTOMAKE
-$AUTOCONF
-./configure
+command_ok_ "aclocal" $ACLOCAL
+command_ok_ "automake" $AUTOMAKE
+command_ok_ "autoconf" $AUTOCONF
+command_ok_ "configure" ./configure
-$MAKE
-$MAKE distcheck
+directive=''; make_can_chain_suffix_rules || directive=TODO
-$MAKE clean
+for target in all distcheck; do
+ command_ok_ "make $target" \
+ -D "$directive" -r "suffix rules not chained" \
+ $MAKE $target
+done
+command_ok_ "clean" $MAKE clean
+# Sanity check.
cat >> Makefile <<'END'
foo.c: foo.c0
foo.c0: foo.c1
END
-
-$MAKE
+command_ok_ "make with explicit dependencies" $MAKE
:
$ACLOCAL
$AUTOMAKE -i
-grep '^ *\.c' Makefile.in # for debugging
+grep '^ *\.c' Makefile.in # For debugging.
test `grep -c '^\.c\.o:' Makefile.in` -eq 1
test `grep -c '^\.c\.obj:' Makefile.in` -eq 1
$AUTOMAKE
-grep '^ *\.c' Makefile.in # for debugging
+grep '^ *\.c' Makefile.in # For debugging.
test `grep -c '^\.c\.o:' Makefile.in` -eq 1
test `grep -c '^\.c\.obj:' Makefile.in` -eq 1
# Make sure that derivations work with .lo too.
# (related to PR/37)
-required='libtoolize yacc'
+required='cc libtoolize yacc'
. ./defs || Exit 1
+plan_ 7
+
cat >>configure.in <<EOF
AC_PROG_CC
AC_PROG_YACC
libfoo_la_SOURCES = foo.x_
.x_.y:
rm -f $@ $@-t
-## the leading `:;' works around a bug in bash <= 3.2
+## The leading `:;' works around a bug in bash <= 3.2.
:; { echo '/* autogenerated */' \
&& echo '%{' \
&& echo 'int yylex () {return 0;}' \
&& echo '%%' \
&& echo 'WORD: "foo";' \
&& echo '%%' \
-## account for VPATH issues on weaker make implementations
+## Account for VPATH issues on weaker make implementations.
&& cat `test -f '$<' || echo $(srcdir)/`$<; \
} > $@-t
mv -f $@-t $@
int foo (void) { return yyparse(); }
END
-libtoolize --force
-$ACLOCAL
-$AUTOCONF
-$AUTOMAKE --add-missing
-
-./configure
-$MAKE test
+command_ok_ "libtoolize" libtoolize --force
+command_ok_ "aclocal" $ACLOCAL
+command_ok_ "autoconf" $AUTOCONF
+command_ok_ "automake" $AUTOMAKE --add-missing
+command_ok_ "configure" ./configure
+command_ok_ "make test" $MAKE test
-make_can_chain_suffix_rules || skip_ "make doesn't chain suffix rules"
-$MAKE all
+directive=''; make_can_chain_suffix_rules || directive=TODO
+command_ok_ "make all" -D "$directive" -r "suffix rules not chained" \
+ $MAKE all
:
# PR/371 and PR/372: Reported by Duncan Gibson.
# We also check for suffixes containing `-'.
+required=cc
. ./defs || Exit 1
+plan_ 10
+
cat >>configure.in <<EOF
AC_PROG_CC
AC_OUTPUT
baz_SOURCES = baz1.x_ baz2.y-z
.y-z.c .x_.c:
-## account for VPATH issues on weaker make implementations (e.g. IRIX 6.5)
+## Account for VPATH issues on weaker make implementations (e.g. IRIX 6.5).
sed 's/INTEGER/int/g' `test -f '$<' || echo $(srcdir)/`$< >$@
CLEANFILES = foo.c bar.c baz1.c baz2.c
echo 'INTEGER main(void) { INTEGER baz(void); return baz(); }' > baz1.x_
echo 'INTEGER baz(void) { return 0; }' > baz2.y-z
-$ACLOCAL
-$AUTOCONF
+command_ok_ "aclocal" $ACLOCAL
+command_ok_ "autoconf" $AUTOCONF
# What we do is not portable. Automake should warn.
AUTOMAKE_fails -a
-grep '[iI]nference rules can have only one target before the colon' stderr
-
+command_ok_ "warn about unportable make usage" \
+ grep '[iI]nference rules can have only one target before the colon' stderr
# But this should work anyway.
-$AUTOMAKE -a -Wno-portability
-
-./configure
+command_ok_ "automake" $AUTOMAKE -a -Wno-portability
-OBJEXT=foo $MAKE -e test-fake
-$MAKE test-real
+command_ok_ "configure" ./configure
+command_ok_ "make test-fake" env OBJEXT=foo $MAKE -e test-fake
+command_ok_ "make test-real" $MAKE test-real
-make_can_chain_suffix_rules || skip_ "make doesn't chain suffix rules"
+directive=''; make_can_chain_suffix_rules || directive=TODO
-$MAKE
-$MAKE distcheck
+for target in '' distcheck; do
+ command_ok_ "make $target" \
+ -D "$directive" -r "suffix rules not chained" \
+ $MAKE $target
+done
:
# Tests that Automake understands suffix rules with subdir objects.
# Reported by John Ratliff.
+required=cc
. ./defs || Exit 1
cat >>configure.in <<EOF
cat >Makefile.am << 'END'
AUTOMAKE_OPTIONS = subdir-objects
SUFFIXES = .baz .o
-# we fake here:
+# We fake here:
.baz.o:
- ## account for VPATH issues on weaker make implementations
+## Account for VPATH issues on weaker make implementations.
cp `test -f '$<' || echo $(srcdir)/`$< $@
bin_PROGRAMS = foo
# and subdir objects.
# Reported by Florian Briegel.
+required=cc
. ./defs || Exit 1
cat >>configure.in <<EOF
SUFFIXES = .baz .c
.baz.c:
case $@ in sub/*) $(MKDIR_P) sub;; *) :;; esac
-## Account for VPATH issues on weaker make implementations (e.g. IRIX 6.5)
+## Account for VPATH issues on weaker make implementations (e.g. IRIX 6.5).
cp `test -f '$<' || echo $(srcdir)/`$< $@
DISTCLEANFILES = sub/bar.c
$MAKE distcheck
$MAKE distclean
-# Should also work without subdir-objects
+# Should also work without subdir-objects.
sed '/subdir-objects/d' < Makefile.am > t
mv -f t Makefile.am
$ACLOCAL
$AUTOMAKE -i
-grep '^ *\.c' Makefile.in # for debugging
+grep '^ *\.c' Makefile.in # For debugging.
test `grep -c '^\.c\.o:' Makefile.in` -eq 1
test `grep -c '^\.c\.obj:' Makefile.in` -eq 1
$AUTOMAKE
-grep '^ *\.c' Makefile.in # for debugging
+grep '^ *\.c' Makefile.in # For debugging.
test `grep -c '^\.c\.o:' Makefile.in` -eq 1
test `grep -c '^\.c\.obj:' Makefile.in` -eq 1
# Test to make sure that suffix rules chain.
+required=c++
. ./defs || Exit 1
+plan_ 10
+
cat >> configure.in << 'END'
AC_PROG_CXX
AC_OUTPUT
CLEANFILES = $(FOO).cc
END
-$ACLOCAL
-$AUTOMAKE
+command_ok_ "aclocal" $ACLOCAL
+command_ok_ "automake" $AUTOMAKE
# The foo.cc intermediate step is implicit, it's a mistake if
# Automake requires this file somewhere. Also, Automake should
# not require the file `foo.c' anywhere.
-$FGREP foo.c Makefile.in && Exit 1
+command_ok_ "intermediate files not mentioned" \
+ not $FGREP foo.c Makefile.in
# However Automake must figure that foo.zoo is eventually
# transformed into foo.o, and use this latter file (to link foo).
-$FGREP 'foo.$(OBJEXT)' Makefile.in
-
-make_can_chain_suffix_rules || skip_ "make doesn't chain suffix rules"
+command_ok_ "final object file figured out" \
+ $FGREP 'foo.$(OBJEXT)' Makefile.in
-$AUTOCONF
-./configure
+command_ok_ "autoconf" $AUTOCONF
+command_ok_ "configure" ./configure
# This is deliberately valid C++, but invalid C.
cat > foo.zoo <<'END'
#include <iostream>
using namespace std;
-INTEGER main(void)
+INTEGER main (void)
{
return 0;
}
END
-$MAKE
+directive=''; make_can_chain_suffix_rules || directive=TODO
+
+for target in all distcheck; do
+ command_ok_ "make $target" \
+ -D "$directive" -r "suffix rules not chained" \
+ $MAKE $target
+done
+
# FIXME: should we check that intermediate file `foo.cc' has
# been removed? Or is this requiring too much from the make
# implementation?
# Intermediate files should not be distributed.
-$MAKE distdir
-test ! -r $me-1.0/foo.cc
-
-# Check the distribution.
-$MAKE distcheck
+command_ok_ "make distdir" $MAKE distdir
+command_ok_ "intermediate file not distributed" test ! -r $me-1.0/foo.cc
:
# from user-defined implicit rules.
# Based on a report from Arkadiusz Miskiewicz <misiek@pld.ORG.PL>.
-required=libtool
+required='cc libtool'
. ./defs || Exit 1
cat >> configure.in << 'END'
cat > Makefile.am << 'END'
.k.lo:
-## account for VPATH issues on weaker make implementations
+## Account for VPATH issues on weaker make implementations.
(echo $< && cat `test -f '$<' || echo $(srcdir)/`$<) > $@
noinst_LTLIBRARIES = libfoo.la
.PHONY: test
test: all
grep '^=GREP=ME=$$' foo.lo
-## weaker regex to account for VPATH issues
+## Weaker regex to account for VPATH issues.
grep 'foo\.k$$' foo.lo
check-local: test
END
: > ltmain.sh
-cp "$testsrcdir/../lib/config.guess" "$testsrcdir/../lib/config.sub" .
+cp "$top_testsrcdir"/lib/config.guess "$top_testsrcdir"/lib/config.sub .
$ACLOCAL
$AUTOMAKE
# foo.$(OBJEXT) using the following rules:
# fooa --[ab]--> foob --[b.$(OBJEXT)]--> foo.$(OBJEXT)
$FGREP ' foo.$(OBJEXT) ' foo-objects
-# barc --[c.o]--> bar.$(OBJEXT) ## This is really meant !
+# barc --[c.o]--> bar.$(OBJEXT) ## This is really meant!
$FGREP ' bar.$(OBJEXT) ' foo-objects
-# bazc --[c.obj]--> baz.$(OBJEXT) ## This is really meant !
+# bazc --[c.obj]--> baz.$(OBJEXT) ## This is really meant!
$FGREP ' baz.$(OBJEXT) ' foo-objects
:
# Test to make sure Automake supports implicit rules with dot-less
# extensions. Se also related "grepping" test suffix6.test.
-required=GNUmake # other makes might not grok dot-less suffix rules
+required=GNUmake # Other makes might not grok dot-less suffix rules.
. ./defs || Exit 1
cat >> configure.in << 'END'
# $(LINK) is not defined automatically by Automake, since the *_SOURCES
-# variables don't contain any known extension (.c, .cc, .f ...),
+# variables don't contain any known extension (.c, .cc, .f, ...).
# So we need this hack -- but since such an hack can also serve as a
# mild stress test, that's ok.
AC_SUBST([LINK], ['cat >$@'])
c.$(OBJEXT):
{ echo '=b.obj=' && cat $<; } >$@
test:
- : for debugging
+ : For debugging.
ls -l
- : implicit intermediate files should be removed by GNU make
+ : Implicit intermediate files should be removed by GNU make ...
test ! -r foob
test ! -r fooc
- : but object files should not
+ : ... but object files should not.
cat foo.$(OBJEXT)
- : for debugging
+ : For debugging.
cat foo.XxX
- : now check that the chain of implicit rules has been executed
- : completely and in the correct order
+ : Now check that the chain of implicit rules has been executed
+ : completely and in the correct order.
(echo =b.obj= && echo =bc= && echo =ab= && echo =src=) > exp
diff exp foo.XxX
rm -f exp
AC_OUTPUT
END
+unset OBJEXT || :
+
cat > Makefile.am << 'END'
SUFFIXES = .zoo .o .obj .@OBJEXT@
# same suffix.
# From PR/37.
-required=libtoolize
+required='cc libtoolize'
. ./defs || Exit 1
+plan_ 10
+
cat >>configure.in <<'END'
AM_PROG_LIBTOOL
AC_OUTPUT
check-local: test1 test2
END
-echo 'int main(void) { return 0; }' > foo.x_
-echo 'int bar(void) { return 0; }' > bar.x_
+echo 'int main (void) { return 0; }' > foo.x_
+echo 'int bar (void) { return 0; }' > bar.x_
-libtoolize
-$ACLOCAL
-$AUTOCONF
-$AUTOMAKE -a
+command_ok_ "libtoolize" libtoolize
+command_ok_ "aclocal" $ACLOCAL
+command_ok_ "autoconf" $AUTOCONF
+command_ok_ "automake" $AUTOMAKE -a
+command_ok_ "configure" ./configure
+command_ok_ "make test0" env OBJEXT=foo $MAKE -e test0
+command_ok_ "make test1" $MAKE test1
-./configure
+directive=''; make_can_chain_suffix_rules || directive=TODO
-OBJEXT=foo $MAKE -e test0
-$MAKE test1
-make_can_chain_suffix_rules || skip_ "make doesn't chain suffix rules"
-$MAKE test2
-$MAKE all
-$MAKE distcheck
+for target in test2 all distcheck; do
+ command_ok_ "make $target" \
+ -D "$directive" -r "suffix rules not chained" \
+ $MAKE $target
+done
:
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Make sure we get an error if symlink creation fails.
-# Reported by Joerg-Martin Schwarz
+# Reported by Joerg-Martin Schwarz.
+am_create_testdir=empty
. ./defs || Exit 1
-rm -f install-sh
-rm -f mkinstalldirs
-
: > Makefile.am
cat > configure.in << 'END'
AC_INIT([symlink], [1.0])
-AC_CONFIG_AUX_DIR(sub)
+AC_CONFIG_AUX_DIR([subdir])
AM_INIT_AUTOMAKE
AC_CONFIG_FILES([Makefile])
END
$ACLOCAL
AUTOMAKE_fails --add-missing
-grep 'error while making link' stderr
+
+grep '^configure\.in:3: .*missing.*error while making link' stderr
+grep '^configure\.in:3: .*install-sh.*error while making link' stderr
+
+test `$FGREP -c 'error while making link' stderr` -eq 2
+
+:
. ./defs || Exit 1
rm -f install-sh
-ln -s Zardoz install-sh || {
- echo "$me: cannot create broken symlinks" >&2
- Exit 77
-}
+ln -s Zardoz install-sh || skip_ "cannot create broken symlinks"
: > Makefile.am
cat > Makefile.am << 'END'
foo = q \
-lib_LTLIBRARIES = foo.la
+bar = z
END
$ACLOCAL
AUTOMAKE_fails
-grep 'Makefile\.am:2:.*blank line' stderr
+grep '^Makefile\.am:2:.*blank line follow.* trailing backslash' stderr
:
bin_PROGRAMS = bar
END
-AUTOMAKE_run 0
+AUTOMAKE_run
grep 'define.*TAGS_DEPENDENCIES.*without' stderr && Exit 1
:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - handling of "ambiguous" TODO and SKIP directives
+# See also related test 'tap-todo-skip-together.test'.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > all.test <<END
+1..6
+ok 1 # foo SKIP
+not ok 2 # bar TODO
+ok 3 # :SKIP
+not ok 4 # :TODO
+ok 5 # SKIP SKIP
+not ok 6 # TODO TODO
+END
+
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=6 pass=2 fail=2 xpass=0 xfail=1 skip=1 error=0
+
+cat > exp << 'END'
+PASS: all.test 1 # foo SKIP
+FAIL: all.test 2 # bar TODO
+PASS: all.test 3 # :SKIP
+FAIL: all.test 4 # :TODO
+SKIP: all.test 5 # SKIP SKIP
+XFAIL: all.test 6 # TODO TODO
+END
+
+$FGREP ': all.test' stdout > got
+
+cat exp
+cat got
+diff exp got
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - unnumbered tests are OK, as long as their final total number
+# agrees with the plan
+# - test results without number get automatically numbered in the
+# console progress output
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > all.test <<'END'
+1..14
+ok
+ok foo
+ok - foo2
+not ok
+not ok bar
+not ok - bar2
+; See that we can intermingle different kind of results without
+; messing up the autonumbering
+ok
+ok # TODO
+not ok # TODO who cares?
+ok
+not ok
+ok muuu # SKIP
+not ok
+ok
+END
+
+TESTS=all.test $MAKE -e check >stdout && { cat stdout; Exit 1; }
+cat stdout
+count_test_results total=14 pass=6 fail=5 xpass=1 xfail=1 skip=1 error=0
+
+cat > exp <<'END'
+PASS: all.test 1
+PASS: all.test 2 foo
+PASS: all.test 3 - foo2
+FAIL: all.test 4
+FAIL: all.test 5 bar
+FAIL: all.test 6 - bar2
+PASS: all.test 7
+XPASS: all.test 8 # TODO
+XFAIL: all.test 9 # TODO who cares?
+PASS: all.test 10
+FAIL: all.test 11
+SKIP: all.test 12 muuu # SKIP
+FAIL: all.test 13
+PASS: all.test 14
+END
+
+$FGREP ': all.test' stdout > got
+
+cat exp
+cat got
+diff exp got
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - missing, unreadable, or not-executable test scripts cause proper
+# error reports
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+plan_ 5
+
+fetch_tap_driver
+
+cat >> configure.in <<END
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+TEST_LOG_DRIVER = $(srcdir)/tap-driver
+TESTS = none.test noread.test noexec.test
+none.test:
+END
+
+cat > noexec.test <<'END'
+#!/bin/sh
+echo 1..1
+ok 1
+END
+
+cp noexec.test noread.test
+chmod a-r noread.test
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE
+
+./configure
+
+if $MAKE check >stdout; then r='not ok'; else r='ok'; fi
+cat stdout
+result_ "$r" '"make check" returns non-zero exit status'
+
+# FIXME: maybe grep for stricter error messages in the next checks?
+
+command_ok_ "non-existent test is reported" \
+ grep '^ERROR: none\.test' stdout
+
+desc="non-executable test is reported"
+if ./noexec.test; then
+ skip_ -r "any file is executable" "$desc"
+else
+ command_ok_ "$desc" -- grep '^ERROR: noexec\.test' stdout
+fi
+
+desc="non-readable test is reported"
+if test -r noread.test; then
+ skip_ -r "any file is readable" "$desc"
+else
+ command_ok_ "$desc" -- grep '^ERROR: noread\.test' stdout
+fi
+
+# Check that no spurious test results is reported. This is lower-priority
+# (and in fact the check currently fails.
+command_ok_ 'no spurious results' -D TODO -r 'still get "missing plan"' \
+ count_test_results total=3 pass=0 fail=0 xpass=0 xfail=0 skip=0 error=3
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - even after a "Bail out!" directive, all input is still copied in
+# the log file
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > all.test <<END
+First line
+Bail out!
+non-TAP line after bailout
+# TAP diagnostic after bailout
+1..0 # SKIP TAP plan after bailout
+ok 1 - TAP result after bailout
+END
+
+$MAKE check && { cat all.log; Exit 1; }
+cat all.log
+
+for rx in \
+ 'First line' \
+ 'Bail out!' \
+ 'non-TAP line after bailout' \
+ '# TAP diagnostic after bailout' \
+ '1\.\.0 # SKIP TAP plan after bailout' \
+ 'ok 1 - TAP result after bailout' \
+; do
+ grep "^$rx$" all.log
+done
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Basic TAP test protocol support:
+# - A "Bail out!" directive causes the driver to ignore the exit
+# status of the test script.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+tests='exit.test exit127.test sighup.test sigterm.test'
+
+cat > Makefile.am <<END
+TESTS = $tests
+END
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > exit.test << 'END'
+#!/bin/sh
+echo 'Bail out!'
+exit 1
+END
+
+cat > exit127.test << 'END'
+#!/bin/sh
+echo 'Bail out!'
+exit 127
+END
+
+cat > sighup.test << 'END'
+#!/bin/sh
+echo 'Bail out!'
+kill -1 $$
+END
+
+cat > sigterm.test << 'END'
+#!/bin/sh
+echo 'Bail out!'
+kill -15 $$
+END
+
+chmod a+x $tests
+
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=4 pass=0 fail=0 xpass=0 xfail=0 skip=0 error=4
+for tst in $tests; do grep "^ERROR: $tst - Bail out!" stdout; done
+$EGREP "ERROR: .*(exit.*status|terminat.*signal)" stdout && Exit 1
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - A "Bail out!" directive causes the driver to ignore any TAP
+# diagnostic message in the rest of the following TAP stream.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+echo AM_TEST_LOG_DRIVER_FLAGS = --comments >> Makefile
+
+cat > all.test <<END
+1..1
+# sanity check
+ok 1
+Bail out!
+# not seen
+END
+
+# Doing the sums above, we have:
+
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=2 pass=1 fail=0 xpass=0 xfail=0 skip=0 error=1
+grep '# all.test: sanity check' stdout
+grep 'not seen' stdout && Exit 1
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - A "Bail out!" directive causes the driver to ignore any TAP
+# result or error in the rest of the following TAP stream.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+# Various errors that can all be squashed into a single test script.
+cat > foo.test << 'END'
+1..5
+Bail out!
+# All possible test results.
+# Test out-of-order.
+ok 4
+# Extra TAP plan.
+1..2
+# Another bailout directive.
+Bail out! Not seen.
+# Stop now, with too few tests run.
+END
+
+# Tests run after a "SKIP" plan.
+cat > bar.test << 'END'
+1..0 # SKIP
+Bail out!
+ok 1
+END
+
+# Too many tests run.
+cat > baz.test << 'END'
+1..1
+ok 1
+Bail out!
+ok 2
+ok 3
+END
+
+TESTS='foo.test bar.test baz.test' $MAKE -e check >stdout \
+ && { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=5 pass=1 fail=0 xpass=0 xfail=0 skip=1 error=3
+
+grep '^ERROR: foo\.test - Bail out!$' stdout
+grep '^ERROR: bar\.test - Bail out!$' stdout
+grep '^SKIP: bar\.test' stdout
+grep '^ERROR: baz\.test - Bail out!$' stdout
+grep '^PASS: baz\.test 1$' stdout
+
+$FGREP 'Not seen' stdout && Exit 1
+
+test `$FGREP -c ': foo.test' stdout` -eq 1
+test `$FGREP -c ': bar.test' stdout` -eq 2
+test `$FGREP -c ': baz.test' stdout` -eq 2
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Basic TAP test protocol support:
+# - "Bail out!" magic
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+: > exp
+
+#------------------------------------------------------------------
+
+# Bailout without explanation.
+
+cat > a.test <<END
+1..4
+ok 1
+not ok 2
+Bail out!
+not ok 3
+ok 4 # SKIP
+END
+
+cat >> exp <<END
+PASS: a.test 1
+FAIL: a.test 2
+ERROR: a.test - Bail out!
+END
+
+# pass += 1, fail +=1, error += 1
+
+#------------------------------------------------------------------
+
+# Bailout with explanation.
+
+cat > b.test <<END
+1..7
+ok 1 # SKIP
+ok 2 # TODO
+not ok 3 # TODO
+Bail out! We're out of disk space.
+ok 4
+not ok 5
+not ok 6 # TODO
+ok 7 # TODO
+END
+
+cat >> exp <<END
+SKIP: b.test 1 # SKIP
+XPASS: b.test 2 # TODO
+XFAIL: b.test 3 # TODO
+ERROR: b.test - Bail out! We're out of disk space.
+END
+
+# skip += 1, xpass += 1, xfail += 1, error += 1
+
+#------------------------------------------------------------------
+
+# Bail out before the test plan.
+
+cat > c.test <<END
+ok 1
+ok 2
+Bail out! BOOOH!
+1..2
+END
+
+cat >> exp <<END
+PASS: c.test 1
+PASS: c.test 2
+ERROR: c.test - Bail out! BOOOH!
+END
+
+# pass += 2, error += 1
+
+#------------------------------------------------------------------
+
+# Bailout on the first line.
+
+cat > d.test <<END
+Bail out! mktemp -d: Permission denied
+ok 1
+END
+
+echo 'ERROR: d.test - Bail out! mktemp -d: Permission denied' >> exp
+
+# error += 1
+
+#------------------------------------------------------------------
+
+# TAP input comprised only of a bailout directive.
+
+cat > e.test <<END
+Bail out!
+END
+
+echo "ERROR: e.test - Bail out!" >> exp
+
+# error += 1
+
+#------------------------------------------------------------------
+
+# Doing the sums above, we have:
+test_counts='total=12 pass=3 fail=1 xpass=1 xfail=1 skip=1 error=5'
+
+TESTS='a.test b.test c.test d.test e.test' $MAKE -e check >stdout \
+ && { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results $test_counts
+
+LC_ALL=C sort exp > t
+mv -f t exp
+
+# We need the sort below to account for parallel make usage.
+grep ': [abcde]\.test' stdout | LC_ALL=C sort > got
+
+cat exp
+cat got
+diff exp got
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Basic TAP support:
+# - LOG_COMPILER support;
+# - basic support for TODO and SKIP directives, and "Bail out!" magic;
+# - testsuite progress output on console;
+# - runtime overriding of TESTS and TEST_LOGS;
+# - correct counts of test results (both in summary and in progress
+# output on console).
+# Note that some of the features checked here are checked in other
+# test cases too, usually in a more thorough and detailed way.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+fetch_tap_driver
+
+cat >> configure.in <<END
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+TEST_LOG_DRIVER = $(srcdir)/tap-driver
+## Defining LOG_COMPILER should work and not intefere with the
+## tap-driver script.
+TEST_LOG_COMPILER = cat
+TESTS = success.test
+
+ok.test:
+ echo '1..3' > $@-t
+ echo 'ok 1' >> $@-t
+ echo 'not ok 2 # TODO' >>$@-t
+ echo 'ok 3 # SKIP' >>$@-t
+ cat $@-t ;: For debugging.
+ mv -f $@-t $@
+END
+
+cat > success.test << 'END'
+1..20
+ok 1
+ok 2 two
+ok 3 - three
+ok 4 four four
+not ok 5
+not ok 6 six
+not ok 7 - seven
+not ok 8 eight eight
+ok 9 # TODO
+ok 10 ten # TODO
+ok 11 - eleven # TODO
+ok 12 twelve twelve # TODO
+not ok 13 # TODO
+not ok 14 fourteen # TODO
+not ok 15 - fifteen # TODO
+not ok 16 sixteen sixteen # TODO
+ok 17 # SKIP
+ok 18 eighteen # SKIP
+ok 19 - nineteen # SKIP
+ok 20 twenty twenty # SKIP
+END
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE
+
+./configure
+
+# Basilar usage and testsuite progress output.
+
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=20 pass=4 fail=4 xpass=4 xfail=4 skip=4 error=0
+
+test -f success.log
+test -f test-suite.log
+
+cat > exp << 'END'
+PASS: success.test 1
+PASS: success.test 2 two
+PASS: success.test 3 - three
+PASS: success.test 4 four four
+FAIL: success.test 5
+FAIL: success.test 6 six
+FAIL: success.test 7 - seven
+FAIL: success.test 8 eight eight
+XPASS: success.test 9 # TODO
+XPASS: success.test 10 ten # TODO
+XPASS: success.test 11 - eleven # TODO
+XPASS: success.test 12 twelve twelve # TODO
+XFAIL: success.test 13 # TODO
+XFAIL: success.test 14 fourteen # TODO
+XFAIL: success.test 15 - fifteen # TODO
+XFAIL: success.test 16 sixteen sixteen # TODO
+SKIP: success.test 17 # SKIP
+SKIP: success.test 18 eighteen # SKIP
+SKIP: success.test 19 - nineteen # SKIP
+SKIP: success.test 20 twenty twenty # SKIP
+END
+
+$FGREP ': success.test' stdout > got
+
+cat exp
+cat got
+diff exp got
+
+# Override TESTS from the command line.
+
+rm -f *.log *.test
+
+cat > bail.test <<'END'
+1..1
+Bail out!
+ok 1
+END
+
+TESTS=bail.test $MAKE -e check >stdout && { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=1 pass=0 fail=0 xpass=0 xfail=0 skip=0 error=1
+
+test ! -f success.log
+test -f bail.log
+test -f test-suite.log
+
+grep '^ERROR: bail\.test - Bail out!' stdout
+grep '^PASS:' stdout && Exit 1
+test `$FGREP -c ': bail.test' stdout` -eq 1
+$FGREP 'success.test' stdout && Exit 1
+
+# Override TEST_LOGS from the command line, making it point to a test
+# (ok.test) that has to be generated at make time.
+
+rm -f *.log *.test
+
+TEST_LOGS=ok.log $MAKE -e check >stdout || { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=3 pass=1 fail=0 xpass=0 xfail=1 skip=1 error=0
+
+test -f ok.test
+test -f ok.log
+test ! -f success.log
+test ! -f bail.log
+test -f test-suite.log
+
+$EGREP '(bail|success)\.test' stdout && Exit 1
+
+cat > exp << 'END'
+PASS: ok.test 1
+XFAIL: ok.test 2 # TODO
+SKIP: ok.test 3 # SKIP
+END
+
+$FGREP ': ok.test' stdout > got
+
+cat exp
+cat got
+diff exp got
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - colorization of TAP results and diagnostic messages
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+TERM=ansi; export TERM
+
+esc='\e'
+# Escape `[' for grep, below.
+red="$esc\[0;31m"
+grn="$esc\[0;32m"
+lgn="$esc\[1;32m"
+blu="$esc\[1;34m"
+mgn="$esc\[0;35m"
+std="$esc\[m"
+
+# Check that grep can parse nonprinting characters.
+# BSD 'grep' works from a pipe, but not a seekable file.
+# GNU or BSD 'grep -a' works on files, but is not portable.
+case `echo "$std" | grep .` in
+ "$std") ;;
+ *) echo "$me: grep can't parse nonprinting characters" >&2; Exit 77;;
+esac
+
+cat > Makefile.am << 'END'
+AUTOMAKE_OPTIONS = color-tests
+AM_TEST_LOG_DRIVER_FLAGS = --comments
+TEST_LOG_COMPILER = cat
+TESTS = all.test skip.test bail.test badplan.test noplan.test \
+ few.test many.test order.test afterlate.test
+END
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > all.test << 'END'
+1..5
+ok 1 - foo
+# Hi! I shouldn't be colorized!
+not ok 2 - bar # TODO td
+ok 3 - baz # SKIP sk
+not ok 4 - quux
+ok 5 - zardoz # TODO
+END
+
+cat > skip.test << 'END'
+1..0 # SKIP whole script
+END
+
+cat > bail.test << 'END'
+1..1
+ok 1
+Bail out!
+END
+
+cat > badplan.test << 'END'
+1..2
+ok 1
+1..2
+ok 2
+END
+
+cat > noplan.test << 'END'
+ok 1
+END
+
+cat > few.test << 'END'
+1..2
+ok 1
+END
+
+cat > many.test << 'END'
+1..1
+ok 1
+ok 2
+END
+
+cat > order.test << 'END'
+1..1
+ok 5
+END
+
+cat > afterlate.test << 'END'
+ok 1
+1..2
+ok 2
+END
+
+AM_COLOR_TESTS=always $MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+
+test_color ()
+{
+ # Not a useless use of cat; see above comments about grep.
+ cat stdout | grep "^${grn}PASS${std}: all\.test 1 - foo$"
+ cat stdout | grep "^${lgn}XFAIL${std}: all\.test 2 - bar # TODO td$"
+ cat stdout | grep "^${blu}SKIP${std}: all\.test 3 - baz # SKIP sk$"
+ cat stdout | grep "^${red}FAIL${std}: all\.test 4 - quux$"
+ cat stdout | grep "^${red}XPASS${std}: all\.test 5 - zardoz # TODO$"
+ cat stdout | grep "^${blu}SKIP${std}: skip\.test - whole script$"
+ cat stdout | grep "^${grn}PASS${std}: bail\.test 1$"
+ cat stdout | grep "^${mgn}ERROR${std}: bail\.test - Bail out!$"
+ cat stdout | grep "^${mgn}ERROR${std}: badplan\.test - multiple test plans$"
+ cat stdout | grep "^${mgn}ERROR${std}: noplan\.test - missing test plan$"
+ cat stdout | grep "^${mgn}ERROR${std}: few.test - too few tests run (expected 2, got 1)$"
+ cat stdout | grep "^${mgn}ERROR${std}: many.test - too many tests run (expected 1, got 2)$"
+ cat stdout | grep "^${mgn}ERROR${std}: many.test 2 # UNPLANNED$"
+ cat stdout | grep "^${mgn}ERROR${std}: order.test 5 # OUT-OF-ORDER (expecting 1)$"
+ cat stdout | grep "^${mgn}ERROR${std}: afterlate\.test 2 # AFTER LATE PLAN$"
+ # Diagnostic messages shouldn't be colorized.
+ cat stdout | grep "^# all\.test: Hi! I shouldn't be colorized!$"
+ :
+}
+
+test_no_color ()
+{
+ # With make implementations that, like Solaris make, in case of errors
+ # print the whole failing recipe on standard output, we should content
+ # ourselves with a laxer check, to avoid false positives.
+ # Keep this in sync with lib/am/check.am:$(am__color_tests).
+ if $FGREP '= Xalways || test -t 1 ' stdout; then
+ # Extra verbose make, resort to laxer checks.
+ # But we also want to check that the testsuite summary is not unduly
+ # colorized.
+ (
+ set +e # In case some grepped regex below isn't matched.
+ # Not a useless use of cat; see above comments about grep.
+ cat stdout | grep "TOTAL.*:"
+ cat stdout | grep "PASS.*:"
+ cat stdout | grep "FAIL.*:"
+ cat stdout | grep "SKIP.*:"
+ cat stdout | grep "XFAIL.*:"
+ cat stdout | grep "XPASS.*:"
+ cat stdout | grep "ERROR.*:"
+ cat stdout | grep "^#"
+ cat stdout | grep 'test.*expected'
+ cat stdout | grep 'test.*not run'
+ cat stdout | grep '===='
+ cat stdout | grep '[Ss]ee .*test-suite\.log'
+ cat stdout | grep '[Tt]estsuite summary'
+ ) | grep "$esc" && Exit 1
+ : For shells with broken 'set -e'
+ else
+ cat stdout | grep "$esc" && Exit 1
+ : For shells with broken 'set -e'
+ fi
+}
+
+AM_COLOR_TESTS=always $MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+test_color
+
+$MAKE -e check >stdout && { cat stdout; Exit 1; }
+cat stdout
+test_no_color
+
+:
-#!/bin/sh
-# Copyright (C) 1996, 2000, 2001, 2002, 2010, 2011 Free Software
-# Foundation, Inc.
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Test for bug reported by François Pinard.
-# If \ is in AC_OUTPUT, automake barfs.
+# Auxiliary test to set up common data used by many tests on TAP support.
+parallel_tests=yes
. ./defs || Exit 1
-cat > configure.in << 'END'
-AC_INIT
-AM_INIT_AUTOMAKE(nonesuch, nonesuch)
-AC_OUTPUT(Makefile \
-zot)
+cat >> configure.in << END
+AC_OUTPUT
END
-: > Makefile.am
-: > zot.in
+cat > Makefile.am << 'END'
+TEST_LOG_DRIVER = $(srcdir)/tap-driver
+TEST_LOG_COMPILER = cat
+TESTS = all.test
+END
$ACLOCAL
$AUTOCONF
./configure
-test -f zot || Exit 1
-ls -a | grep '\\' && Exit 1
+rm -rf autom4te*.cache
+
+# So that the data files we've created won't be removed at exit.
+keep_testdirs=yes
:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Basic TAP test protocol support:
+# - dependencies between test scripts
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat > Makefile.am << 'END'
+# The tests are *deliberately* listed in inversed order here.
+TESTS = c.test b.test a.test
+b.log: a.log
+c.log: b.log
+END
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > a.test << 'END'
+#!/bin/sh
+echo 1..2
+echo ok 1
+sleep '3' # FIXME: quotes to please maintainer-check
+echo ok 2
+: > a.run
+END
+
+cat > b.test << 'END'
+#!/bin/sh
+echo 1..2
+if test -f a.run; then
+ echo ok 1
+else
+ echo not ok 1
+fi
+sleep '3' # FIXME: quotes to please maintainer-check
+echo ok 2
+: > b.run
+END
+
+cat > c.test << 'END'
+#!/bin/sh
+echo 1..1
+test -f b.run || { echo 'Bail out!'; exit 1; }
+echo ok 1
+rm -f a.run b.run
+END
+
+chmod a+x *.test
+
+$MAKE check >stdout || { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=5 pass=5 fail=0 xpass=0 xfail=0 skip=0 error=0
+
+cat > exp << 'END'
+PASS: a.test 1
+PASS: a.test 2
+PASS: b.test 1
+PASS: b.test 2
+PASS: c.test 1
+END
+
+grep ': [abc]\.test' stdout > got
+
+cat exp
+cat got
+diff exp got
+
+# TODO: it would be nice to also redo the checks forcing parallel make...
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - option '--diagnostic-string' to customize the string introducing
+# TAP diagnostics
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+fetch_tap_driver
+
+cat >> configure.in <<END
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+my_log_driver = $(srcdir)/tap-driver
+my_log_compiler = cat
+TEST_EXTENSIONS =
+TESTS =
+END
+
+: > later.mk
+
+# Quoting our comments below is an hack required to keep
+# comments near the things they refer to.
+i=0
+for string in \
+'## A letter' \
+ a \
+'## A number' \
+ 1023 \
+'## A non-alphabetic character' \
+ @ \
+'## Some metacharacters (we need to repeat the "$" for make)' \
+ '^>;&*"|$$' \
+'## A whitespace character' \
+ " " \
+'## A tab character' \
+ "$tab" \
+'## A string with more whitespace' \
+ " ${tab}a b${tab} c" \
+'## Note the we do not have the empty string here. We prefer to' \
+'## leave its behaviour in this context undefined for the moment.'
+do
+ case $string in '##'*) continue;; esac
+ i=`expr $i + 1`
+ unindent >> Makefile.am << END
+ TEST_EXTENSIONS += .t$i
+ TESTS += foo$i.t$i
+ T${i}_LOG_COMPILER = \$(my_log_compiler)
+ T${i}_LOG_DRIVER = \$(my_log_driver)
+ AM_T${i}_LOG_DRIVER_FLAGS = \
+ --comments \
+ --diagnostic-string '$string'
+END
+ unindent > foo$i.t$i <<END
+ 1..1
+ ok 1
+ $string blah blah $i
+END
+ echo "AM_T${i}_LOG_DRIVER_FLAGS = --no-comments" >> later.mk
+done
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE
+
+./configure
+
+$MAKE check >stdout || { cat stdout; Exit 1; }
+cat stdout
+count_test_results total=$i pass=$i fail=0 xpass=0 xfail=0 skip=0 error=0
+
+cat later.mk >> Makefile
+$MAKE check >stdout || { cat stdout; Exit 1; }
+cat stdout
+$FGREP 'blah blah' stdout && Exit 1
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - diagnostic messages (TAP lines with leading "#")
+# - flags '--comments' and '--no-comments' of the TAP test driver
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+metacharacters=\''"\$!&()[]<>#;^?*'
+
+cat > all.test <<END
+1..4
+# Hi! I'm a comment.
+# Tests begin.
+ok 1
+not ok 2 - foo # TODO
+ok 3 - bar # SKIP
+# Tests end.
+ok - zardoz
+# Shell metacharacters here: $metacharacters
+.# Leading characters before "#", not a TAP diagnostic line.
+x # Leading characters before "#", not a TAP diagnostic line.
+ # Leading whitespace before "#", not a TAP diagnostic line.
+${tab}# Leading whitespace before "#", not a TAP diagnostic line.
+ ${tab} # Leading whitespace before "#", not a TAP diagnostic line.
+END
+
+cat > exp <<END
+# all.test: Hi! I'm a comment.
+# all.test: Tests begin.
+PASS: all.test 1
+XFAIL: all.test 2 - foo # TODO
+SKIP: all.test 3 - bar # SKIP
+# all.test: Tests end.
+PASS: all.test 4 - zardoz
+# all.test: Shell metacharacters here: $metacharacters
+END
+
+$MAKE check >stdout || { cat stdout; Exit 1; }
+cat stdout
+$EGREP -i "#.*all\\.test|a comment|(Tests|Shell) " stdout && Exit 1
+count_test_results total=4 pass=2 fail=0 xpass=0 xfail=1 skip=1 error=0
+
+echo 'AM_TEST_LOG_DRIVER_FLAGS = --comments' >> Makefile
+$MAKE check >stdout || { cat stdout; Exit 1; }
+cat stdout
+$FGREP ' all.test' stdout > got
+cat exp
+cat got
+diff exp got
+count_test_results total=4 pass=2 fail=0 xpass=0 xfail=1 skip=1 error=0
+
+TEST_LOG_DRIVER_FLAGS="--no-comments" $MAKE -e check >stdout \
+ || { cat stdout; Exit 1; }
+cat stdout
+$EGREP -i "#.*all\\.test|a comment|(Tests|Shell) " stdout && Exit 1
+count_test_results total=4 pass=2 fail=0 xpass=0 xfail=1 skip=1 error=0
+
+# The "#"-prepended lines here shouldn't be parsed as test results.
+cat > all.test <<END
+1..1
+ok
+# ok
+#ok
+# not ok
+#not ok
+# Bail out!
+#Bail out!
+# SKIP
+#SKIP
+# TODO
+#TODO
+END
+
+$MAKE check >stdout || { cat stdout; Exit 1; }
+cat stdout
+count_test_results total=1 pass=1 fail=0 xpass=0 xfail=0 skip=0 error=0
+
+# Diagnostic without leading whitespace, or with extra leading whitespace,
+# is ok. Be laxer in the grepping checks, to allow for whitespace
+# normalization by the TAP driver.
+
+ws="[ $tab]"
+ws0p="${ws}*"
+ws1p="${ws}${ws0p}"
+
+cat > all.test <<END
+1..1
+ok 1
+#foo
+#bar${tab}
+# zardoz ${tab}
+# ${tab} ${tab}${tab}foo bar${tab}baz ${tab}
+END
+
+$MAKE check >stdout || { cat stdout; Exit 1; }
+cat stdout
+count_test_results total=1 pass=1 fail=0 xpass=0 xfail=0 skip=0 error=0
+
+grep "^# all.test:${ws0p}foo$" stdout
+grep "^# all.test:${ws0p}bar${ws0p}$" stdout
+grep "^# all.test:${ws1p}zardoz${ws0p}$" stdout
+grep "^# all.test:${ws1p}foo bar${tab}baz${ws0p}$" stdout
+
+test `grep -c '^# all\.test:' stdout` -eq 4
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check that an example given in the documentation really works.
+# See section "Simple Tests" subsection "Script-based Testsuites".
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+fetch_tap_driver
+
+cat >> configure.in <<END
+AC_PROG_CC
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+TESTS = foo.sh zardoz.tap bar.sh mu.tap
+TEST_EXTENSIONS = .sh .tap
+TAP_LOG_DRIVER = $(srcdir)/tap-driver
+END
+
+cat > foo.sh <<'END'
+#!/bin/sh
+exit 0
+END
+
+cat > bar.sh <<'END'
+#!/bin/sh
+exit 77
+END
+
+cat > zardoz.tap << 'END'
+#!/bin/sh
+echo 1..4
+echo 'ok 1 - Daemon started'
+echo 'ok 2 - Daemon responding'
+echo 'ok 3 - Daemon uses /proc # SKIP /proc is not mounted'
+echo 'ok 4 - Daemon stopped'
+END
+
+cat > mu.tap << 'END'
+#!/bin/sh
+echo 1..2
+echo 'ok'
+echo 'not ok # TODO frobnication not yet implemented'
+END
+
+chmod a+x *.sh *.tap
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+./configure
+
+$MAKE check >stdout || { cat stdout; Exit 1; }
+cat stdout
+
+cat > exp <<'END'
+PASS: foo.sh
+PASS: zardoz.tap 1 - Daemon started
+PASS: zardoz.tap 2 - Daemon responding
+SKIP: zardoz.tap 3 - Daemon uses /proc # SKIP /proc is not mounted
+PASS: zardoz.tap 4 - Daemon stopped
+SKIP: bar.sh
+PASS: mu.tap 1
+XFAIL: mu.tap 2 # TODO frobnication not yet implemented
+END
+
+sed -n '/^PASS: foo\.sh/,/^XFAIL: mu\.tap/p' stdout > got
+
+cat exp
+cat got
+diff exp got
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check that an example given in the documentation really works.
+# See section "Using the TAP test protocol", subsection "Use TAP
+# with the Automake test harness".
+
+am_create_testdir=empty
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat > Makefile.am <<'END'
+TEST_LOG_DRIVER = $(PERL) $(srcdir)/build-aux/tap-driver.pl
+TESTS = foo.test bar.test baz.test
+EXTRA_DIST = $(TESTS)
+END
+
+cat > configure.ac <<'END'
+AC_INIT([GNU Try Tap], [1.0], [bug-automake@gnu.org])
+AC_CONFIG_AUX_DIR([build-aux])
+AM_INIT_AUTOMAKE([foreign parallel-tests -Wall -Werror])
+AC_CONFIG_FILES([Makefile])
+AC_REQUIRE_AUX_FILE([tap-driver.pl])
+AC_PATH_PROG([PERL], [perl])
+test -n "$PERL" || AC_MSG_ERROR([perl not found])
+$PERL -MTAP::Parser -e 1 || AC_MSG_ERROR([TAP::Parser not found])
+AC_OUTPUT
+END
+
+cat > foo.test <<'END'
+#!/bin/sh
+echo 1..4 # Number of tests to be executed.
+echo 'ok 1 - Swallows fly'
+echo 'not ok 2 - Caterpillars fly # TODO metamorphosis in progress'
+echo 'ok 3 - Pigs fly # SKIP not enough acid'
+echo '# I just love word plays...'
+echo 'ok 4 - Flies fly too :-)'
+END
+
+cat > bar.test <<'END'
+#!/bin/sh
+echo 1..3
+echo 'not ok 1 - Bummer, this test has failed.'
+echo 'ok 2 - This passed though.'
+echo 'Bail out! Ennui kicking in, sorry...'
+echo 'ok 3 - This will not be seen.'
+END
+
+cat > baz.test <<'END'
+#!/bin/sh
+echo 1..1
+echo ok 1
+# Exit with error, even if all the test case has been successful.
+exit 7
+END
+
+chmod a+x *.test
+
+mkdir build-aux
+cp "$top_testsrcdir"/lib/tap-driver.pl build-aux \
+ || framework_failure_ "fetching the perl TAP driver"
+
+(export AUTOMAKE ACLOCAL AUTOCONF && $AUTORECONF -vi) || Exit 1
+
+./configure --help # Sanity check.
+./configure || skip_ "configure failed"
+
+case $MAKE in *\ -j*) skip_ "can't work easily with concurrent make";; esac
+
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+
+cat > exp <<'END'
+PASS: foo.test 1 - Swallows fly
+XFAIL: foo.test 2 - Caterpillars fly # TODO metamorphosis in progress
+SKIP: foo.test 3 - Pigs fly # SKIP not enough acid
+PASS: foo.test 4 - Flies fly too :-)
+FAIL: bar.test 1 - Bummer, this test has failed.
+PASS: bar.test 2 - This passed though.
+ERROR: bar.test - Bail out! Ennui kicking in, sorry...
+PASS: baz.test 1
+ERROR: baz.test - exited with status 7
+END
+
+sed -n '/^PASS: foo\.test/,/^ERROR: baz\.test/p' stdout > got
+
+cat exp
+cat got
+diff exp got
+
+grep '^Please report to bug-automake@gnu\.org$' stdout
+
+env \
+ TESTS='foo.test baz.test' \
+ TEST_LOG_DRIVER_FLAGS='--comments --ignore-exit' \
+ $MAKE -e check >stdout || { cat stdout; Exit 1; }
+
+cat > exp <<'END'
+PASS: foo.test 1 - Swallows fly
+XFAIL: foo.test 2 - Caterpillars fly # TODO metamorphosis in progress
+SKIP: foo.test 3 - Pigs fly # SKIP not enough acid
+# foo.test: I just love word plays...
+PASS: foo.test 4 - Flies fly too :-)
+PASS: baz.test 1
+END
+
+sed -n '/^PASS: foo\.test/,/^PASS: baz\.test/p' stdout > got
+
+cat exp
+cat got
+diff exp got
+
+# Sanity check the distribution.
+cat > bar.test <<'END'
+#!/bin/sh
+echo 1..1
+echo ok 1
+END
+echo AM_TEST_LOG_DRIVER_FLAGS = --ignore-exit >> Makefile.in
+./config.status Makefile
+$MAKE distcheck
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Unit test on TAP driver:
+# - error messages from awk/shell/perl goes to the console
+
+required=non-root
+parallel_tests=yes
+. ./defs || Exit 1
+
+fetch_tap_driver
+
+tst=zardoz
+
+for suf in trs log; do
+
+ rm -f $tst.log $tst.trs
+ touch $tst.$suf
+ chmod a-w $tst.$suf
+
+ st=0
+ ./tap-driver --test-name $tst --log-file $tst.log --trs-file $tst.trs \
+ -- sh -c 'echo 1..1; echo ok 1; echo "Hello, World!"' \
+ >stdout 2>stderr && st=1
+ cat stdout
+ cat stderr >&2
+ cat $tst.log || :
+ cat $tst.trs || :
+ test $st -eq 0
+
+ $FGREP 'Hello, World!' stderr stdout && Exit 1
+ $FGREP $tst.$suf stderr
+
+done
+
+:
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Sanity check for the automake testsuite.
-# Make sure that $me gets automatically defined by `./defs'.
+# TAP support:
+# - empty diagnostic messages are discarder
-. ./defs-static || exit 1
+parallel_tests=yes
+. ./defs || Exit 1
-set -ex
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
-$SHELL -c '. ./defs && echo me=$me' foo-bar-.test | grep '^me=foo-bar-$'
-$SHELL -c '. ./defs && echo me=$me' _foo__bar.test | grep '^me=_foo__bar$'
-$SHELL -c '. ./defs && echo me=$me' 012.test | grep '^me=012$'
-$SHELL -c '. ./defs && echo me=$me' foo.bar | grep '^me=foo\.bar$'
-$SHELL -c '. ./defs && echo me=$me' a.b.c.test | grep '^me=a\.b\.c$'
+sed 's/\$$//' > all.test <<END
+1..1$
+ok 1$
+#$
+# $
+#${tab}$
+# ${tab} $tab${tab}$
+END
+
+$MAKE check >stdout || { cat stdout; Exit 1; }
+cat stdout
+
+grep '^PASS:.*all\.test' stdout # Sanity check.
+grep '#.*all\.test' stdout && Exit 1
+grep "all\.test[ $tab]*:[ $tab]*$" stdout && Exit 1
:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support: the following situations should be flagged as errors:
+# - empty TAP input
+# - blank TAP input
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+# Empty TAP input.
+: > empty.test
+
+# Blank TAP input (one empty line).
+echo > blank.test
+
+# Blank TAP input (one whitespace-only line).
+cat > white.test <<END
+ ${tab}
+END
+
+# Blank TAP input (few blank and whitespace-only lines).
+cat > white2.test <<END
+
+
+ ${tab} ${tab}${tab}
+
+${tab}
+
+END
+
+for input in empty blank white white2; do
+ cp $input.test all.test
+ $MAKE check >stdout && { cat stdout; Exit 1; }
+ cat stdout
+ count_test_results total=1 pass=0 fail=0 xpass=0 xfail=0 skip=0 error=1
+ grep '^ERROR: all\.test - missing test plan$' stdout
+done
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - "escape" TODO and SKIP directives (by escaping the "#" character)
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > all.test <<'END'
+1..8
+
+not ok \ # TODO
+ok \ # SKIP
+
+not ok \\# TODO
+ok \\# SKIP
+
+ok \\\# TODO
+ok \\\# SKIP
+
+not ok \\\\\\\\\\# TODO
+ok \\\\\\\\\\# SKIP
+END
+
+$MAKE check >stdout || { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=8 pass=2 fail=0 xpass=0 xfail=3 skip=3 error=0
+
+grep '^XFAIL: all\.test 1 .*# TODO' stdout
+grep '^SKIP: all\.test 2 .*# SKIP' stdout
+grep '^XFAIL: all\.test 3 .*# TODO' stdout
+grep '^SKIP: all\.test 4 .*# SKIP' stdout
+grep '^PASS: all\.test 5 .*# TODO' stdout
+grep '^PASS: all\.test 6 .*# SKIP' stdout
+grep '^XFAIL: all\.test 7 .*# TODO' stdout
+grep '^SKIP: all\.test 8 .*# SKIP' stdout
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - "escape" TODO and SKIP directives (by escaping the "#" character)
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > all.test <<'END'
+1..2
+ok \# TODO
+ok \# SKIP
+END
+
+$MAKE check >stdout || { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=2 pass=2 fail=0 xpass=0 xfail=0 skip=0 error=0
+
+grep '^PASS: all\.test 1 .*# TODO' stdout
+grep '^PASS: all\.test 2 .*# SKIP' stdout
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - an exit status != 0 of a test script causes an hard error
+# - the `--ignore-exit' option causes the TAP test driver to ignore
+# exit statuses of the test scripts.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+echo TESTS = > Makefile.am
+for st in 1 2 77 99; do
+ unindent > exit${st}.test <<END
+ #!/bin/sh
+ echo 1..1
+ echo ok 1
+ exit $st
+END
+ echo TESTS += exit${st}.test >> Makefile.am
+done
+
+chmod a+x *.test
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+count_test_results total=8 pass=4 fail=0 xpass=0 xfail=0 skip=0 error=4
+
+grep '^ERROR: exit1\.test - exited with status 1$' stdout
+grep '^ERROR: exit2\.test - exited with status 2$' stdout
+grep '^ERROR: exit77\.test - exited with status 77$' stdout
+grep '^ERROR: exit99\.test - exited with status 99$' stdout
+
+echo TEST_LOG_DRIVER_FLAGS = --ignore-exit >> Makefile
+$MAKE check >stdout || { cat stdout; Exit 1; }
+cat stdout
+count_test_results total=4 pass=4 fail=0 xpass=0 xfail=0 skip=0 error=0
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support: some unusual forms for valid TAP input.
+# See also related test 'tap-fancy2.test'.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+#
+# From manpage Test::Harness::TAP(3):
+#
+# ``Lines written to standard output matching /^(not )?ok\b/ must be
+# interpreted as test lines. All other lines must not be considered
+# test output.''
+#
+# Unfortunately, the exact format of TODO and SKIP directives is not as
+# clearly described in that manpage; but a simple reverse-engineering of
+# the prove(1) utility shows that it is probably given by the perl regex
+# /#\s*(TODO|SKIP)\b/.
+#
+
+cat > all.test <<END
+1..21
+
+ok? a question
+not ok? a question
+
+ok+plus
+not ok+plus
+
+ok-minus
+not ok-minus
+
+ok#55
+not ok#55
+
+ok${tab} ${tab}9
+ok ${tab}${tab} 10
+
+not ok${tab} ${tab}11
+not ok ${tab}${tab} 12
+
+ok# SKIP
+ok${tab}#SKIP--who cares?
+ok?#SKIP!
+ok!#SKIP?
+
+not ok# TODO
+not ok${tab}#TODO--who cares?
+not ok?#TODO!
+not ok!#TODO?
+
+ok~#TODO
+END
+
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=21 pass=6 fail=6 xfail=4 xpass=1 skip=4 error=0
+
+#
+# "Weird" characters support.
+#
+
+# The "#" character might cause confusion w.r.t. TAP directives (TODO,
+# SKIP), so we don't attempt to use it.
+weirdchars=\''"$!&()[]<>;^?*/@%=,.:'
+
+cat > all.test <<END
+1..6
+ok $weirdchars
+not ok $weirdchars
+ok $weirdchars # TODO
+not ok $weirdchars # TODO
+ok $weirdchars # SKIP
+Bail out! $weirdchars
+END
+
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=6 pass=1 fail=1 xfail=1 xpass=1 skip=1 error=1
+$FGREP "PASS: all.test 1 $weirdchars" stdout
+$FGREP "FAIL: all.test 2 $weirdchars" stdout
+$FGREP "XPASS: all.test 3 $weirdchars" stdout
+$FGREP "XFAIL: all.test 4 $weirdchars" stdout
+$FGREP "SKIP: all.test 5 $weirdchars" stdout
+$FGREP "ERROR: all.test - Bail out! $weirdchars" stdout
+
+#
+# Trailing backslashes does not confuse the parser.
+#
+
+bs='\'
+
+cat > all.test <<END
+1..6
+ok $bs
+not ok $bs
+ok # TODO $bs
+not ok # TODO $bs
+ok # SKIP $bs
+Bail out! $bs
+END
+
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=6 pass=1 fail=1 xfail=1 xpass=1 skip=1 error=1
+grep '^PASS: all\.test 1 \\$' stdout
+grep '^FAIL: all\.test 2 \\$' stdout
+grep '^XPASS: all\.test 3 # TODO \\$' stdout
+grep '^XFAIL: all\.test 4 # TODO \\$' stdout
+grep '^SKIP: all\.test 5 # SKIP \\$' stdout
+grep '^ERROR: all\.test - Bail out! \\$' stdout
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support: more unusual forms for valid TAP input.
+# See also related test 'tap-fancy.test'.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+#
+# From manpage Test::Harness::TAP(3):
+#
+# ``Lines written to standard output matching /^(not )?ok\b/ must be
+# interpreted as test lines. All other lines must not be considered
+# test output.''
+#
+# Unfortunately, the exact format of TODO and SKIP directives is not as
+# clearly described in that manpage; but a simple reverse-engineering of
+# the prove(1) utility shows that it is probably given by the perl regex
+# /#\s*(TODO|SKIP)\b/.
+#
+
+# To avoid problems with backslashes in echo arguments.
+xecho () { printf '%s\n' "$*"; }
+
+# There are 34 values for $str ...
+for str in \
+ \' \
+ '"' \
+ '`' \
+ '#' \
+ '$' \
+ '!' \
+ '\' \
+ '/' \
+ '&' \
+ '%' \
+ '(' \
+ ')' \
+ '|' \
+ '^' \
+ '~' \
+ '?' \
+ '*' \
+ '+' \
+ '-' \
+ ',' \
+ ':' \
+ ';' \
+ '=' \
+ '<' \
+ '>' \
+ '@' \
+ '[' \
+ ']' \
+ '{' \
+ '}' \
+ '\\' \
+ '...' \
+ '?[a-zA-Z0-9]*' \
+ '*.*' \
+; do
+ # ... each of them add 1 pass, 1 fail, ...
+ xecho "ok${str}"
+ xecho "not ok${str}"
+ # ... and (generally) 4 skips, 4 xfails, and 4 xpasses ...
+ for settings in \
+ 'result="ok" directive=SKIP' \
+ 'result="not ok" directive=TODO' \
+ 'result="ok" directive=TODO' \
+ ; do
+ eval "$settings"
+ xecho "${result}# ${directive}${str}"
+ # ... but 6 skips, 6 xpasses and 6 xfails are to be removed, since
+ # they might not work with $str = '#' or $str = '\' ...
+ if test x"$str" != x'#' && test x"$str" != x'\'; then
+ xecho "${result}${str}#${directive}"
+ xecho "${result}${str}# ${tab}${tab} ${directive}"
+ xecho "${result}${str}#${directive}${str}"
+ fi
+ done
+done > all.test
+
+# Sanity check against a previous use of unportable usages of backslashes
+# with the "echo" builtin.
+if grep '[^\\]\\#' all.test; then
+ framework_failure_ "writing backslashes in all.test"
+fi
+
+# ... so that we finally have:
+pass=34
+fail=34
+xfail=130 # = 4 * 34 - 6
+xpass=130 # = 4 * 34 - 6
+skip=130 # = 4 * 34 - 6
+error=0
+total=`expr $pass + $fail + $xfail + $xpass + $skip`
+
+# Even nastier! But accordingly to the specifics, it should still work.
+for result in 'ok' 'not ok'; do
+ echo "${result}{[(<#${tab}TODO>)]}" >> all.test
+done
+echo "ok{[(<#${tab}SKIP>)]}" >> all.test
+
+# We have to update some test counts.
+xfail=`expr $xfail + 1`
+xpass=`expr $xpass + 1`
+skip=`expr $skip + 1`
+total=`expr $total + 3`
+
+# And add the test plan!
+echo 1..$total >> all.test
+
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+
+$EGREP '^(PASS|FAIL|SKIP).*#.*TODO' stdout && Exit 1
+$EGREP '^X?(PASS|FAIL).*#.*SKIP' stdout && Exit 1
+
+count_test_results total=$total pass=$pass fail=$fail skip=$skip \
+ xpass=$xpass xfail=$xfail error=$error
+
+:
--- /dev/null
+# -*- shell-script -*-
+#
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Helper functions used by TAP-producing tests of the Automake testsuite.
+
+#
+# IMPORTANT: All the functions defined in this file can *not* be used
+# from within a subshell, unless explicitly noted otherwise.
+#
+
+# The counts of the TAP test results seen so far: total count and
+# per-result counts.
+tap_count_=0
+tap_pass_count_=0
+tap_skip_count_=0
+tap_fail_count_=0
+tap_xfail_count_=0
+tap_xpass_count_=0
+
+# The first "test -n" tries to avoid extra forks when possible.
+if test -n "${ZSH_VERSION}${BASH_VERSION}" \
+ || (eval 'test $((1 + 1)) = 2') >/dev/null 2>&1
+then
+ # Outer use of 'eval' needed to protect dumber shells from parsing
+ # errors.
+ eval 'incr_ () { eval "$1=\$((\${$1} + 1))"; }'
+else
+ incr_ () { eval "$1=\`expr \${$1} + 1\`"; }
+fi
+
+# not COMMAND [ARGS...]
+# ---------------------
+# Run the given command and invert its exit status.
+not ()
+{
+ if "$@"; then return 1; else return 0; fi
+}
+
+# plan_ [unknown|later|lazy|now|NUMBER-OF-PLANNED-TESTS]
+# ------------------------------------------------------
+# Print a TAP plan for the given number of tests. This must be called
+# before reporting any test result. If called with the special argument
+# "unknown" or "later", it will do nothing, expecting the calling script
+# to declare the plan later. If called with the special argument "lazy"
+# or "now", it will print a TAP plan that accounts for the number of tests
+# seen so far.
+plan_ ()
+{
+ if test $# -eq 0; then
+ bailout_ "plan_: missing argument"
+ elif test $# -ge 2; then
+ bailout_ "plan_: too many arguments"
+ elif test x"$1" = x"unknown" || test x"$1" = x"later"; then
+ : No-op.
+ elif test x"$1" = x"lazy" || test x"$1" = x"now"; then
+ echo "1..$tap_count_" # Number of test results seen so far.
+ have_tap_plan_=yes
+ elif test $1 -ge 0; then
+ echo "1..$1"
+ have_tap_plan_=yes
+ else
+ bailout_ "plan_: invalid argument '$1'"
+ fi
+}
+have_tap_plan_=no # Avoid interferences from the environment.
+
+# diag_ [EXPLANATION]
+# ------------------
+# Report the given text as TAP diagnostic. Assumes the string denoting
+# TAP diagnostic lines is stored in the `$diag_string_' variable; this is
+# done to allow better interplay with TAP drivers that allow such a string
+# to be configured.
+diag_ ()
+{
+ test $# -eq 0 || echo "$diag_string_ $*"
+}
+
+# Used by the `diag_' function above. User-overridable.
+diag_string_="#"
+
+# warn_ [EXPLANATION]
+# ------------------
+# Give a warning (using TAP diagnostic).
+warn_ ()
+{
+ diag_ "WARNING:" ${1-"(unknown warning)"} ${1+"$@"}
+}
+
+# result_ RESULT [-D DIRECTIVE] [-r REASON] [--] [DESCRIPTION...]
+# ---------------------------------------------------------------
+# Report a test case with the given RESULT (valid values are "ok" and
+# "not ok") and the given DESCRIPTION (if any). If DIRECTIVE is given
+# and non-empty (valid values being "TODO" and "SKIP"), it will be
+# reported too, with the REASON (if given) appended.
+result_ ()
+{
+ set +x # Don't pollute the log files.
+ test $# -gt 0 || bailout_ "result_: missing argument"
+ tap_result_=$1; shift
+ case $tap_result_ in
+ "ok"|"not ok") ;;
+ *) bailout_ "result_: invalid result '$tap_result'" ;;
+ esac
+ tap_directive_= tap_reason_=
+ while test $# -gt 0; do
+ case $1 in
+ -D|--directive) tap_directive_=$2; shift;;
+ -r|--reason) tap_reason_=$2; shift;;
+ --) shift; break;;
+ -*) bailout_ "result_: invalid option '$1'";;
+ *) break;;
+ esac
+ shift
+ done
+ case $tap_directive_ in
+ ""|TODO|SKIP) ;;
+ *) bailout_ "result_: invalid directive '$directive_'" ;;
+ esac
+ incr_ tap_count_
+ case $tap_result_,$tap_directive_ in
+ ok,) incr_ tap_pass_count_;; # Passed.
+ not\ ok,TODO) incr_ tap_xfail_count_;; # Expected failure.
+ not\ ok,*) incr_ tap_fail_count_ ;; # Failed.
+ ok,TODO) incr_ tap_xpass_count_ ;; # Unexpected pass.
+ ok,SKIP) incr_ tap_skip_count_ ;; # Skipped.
+ *) bailout_ "internal error in 'result_'";; # Can't happen.
+ esac
+ tap_text_="$tap_result_ $tap_count_"
+ if test x"$*" != x; then
+ tap_text_="$tap_text_ - $*"
+ fi
+ if test x"$tap_directive_" != x; then
+ tap_text_="$tap_text_ # $tap_directive_"${tap_reason_:+" $tap_reason_"}
+ fi
+ printf '%s\n' "$tap_text_"
+ set -x # Restore shell xtraces.
+}
+
+# Shorthands for common usages of `result_'.
+ok_ () { result_ 'ok' ${1+"$@"}; }
+not_ok_ () { result_ 'not ok' ${1+"$@"}; }
+skip_ () { result_ 'ok' -D SKIP ${1+"$@"}; }
+
+# skip_row_ COUNT [-r REASON] [--] [DESCRIPTION...]
+# -------------------------------------------------
+# Report a COUNT of skipped test, with the given reason and descriptions
+# (if any). Useful to avoid cascade failures in case a fair number of
+# tests depend on an earlier one that failed.
+skip_row_ ()
+{
+ skip_count_=$1; shift
+ for i_ in `seq $skip_count_`; do skip_ ${1+"$@"}; done
+}
+
+# skip_all_ [REASON ...]
+# ----------------------
+# Skip all the tests in a test script. Must be used before calling `plan_'
+# or reporting any test result. Can't be used from within a subshell.
+skip_all_ ()
+{
+ echo "1..0 # SKIP" ${1+"$@"}
+ have_tap_plan_=yes
+ Exit 0
+}
+
+# bailout_ [REASON ...]
+# ---------------------
+# Stop the execution of the current test suite right now, due to an
+# unrecoverable error. Can be called at any point, but cannot be used
+# from within a subshell.
+bailout_ ()
+{
+ echo 'Bail out!' ${1+"$@"}
+ Exit 99
+}
+
+# fatal_ [REASON ...]
+# -------------------
+# Same as `bailout_'; for compatibility with `plain-functions.sh'.
+fatal_ ()
+{
+ bailout_ ${1+"$@"}
+}
+
+# framework_failure_ [REASON ...]
+# -------------------------------
+# Stop the execution of the current test suite right now, due to an
+# unrecoverable error in the set-up of the test case. Can be called
+# at any point, but cannot be used from within a subshell.
+framework_failure_ ()
+{
+ bailout_ "set-up failure"${1+": $*"}
+}
+
+# command_ok_ TEST-DESCRIPTION [OPTIONS..] [--] CMD [ARGS...]
+# -----------------------------------------------------------
+# Helper subroutine for when a TAP result must be determined by the
+# outcome of a command.
+command_ok_ ()
+{
+ tap_directive_= tap_reason_=
+ test $# -gt 0 || bailout_ "command_ok_: missing argument"
+ tap_description_=$1; shift
+ while test $# -gt 0; do
+ case $1 in
+ -D|--directive) tap_directive_=$2; shift;;
+ -r|--reason) tap_reason_=$2; shift;;
+ --) shift; break;;
+ -*) bailout_ "command_ok_: invalid option '$1'";;
+ *) break;;
+ esac
+ shift
+ done
+ tap_result_="ok"; "$@" || tap_result_="not ok"
+ result_ "$tap_result_" -D "$tap_directive_" -r "$tap_reason_" \
+ -- "$tap_description_"
+}
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - which log files get copied in the global log?
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > ok.test << 'END'
+1..5
+ok 1
+ok 2
+ok 3
+not seen in global log
+ok 4
+ok 5
+END
+
+cat > top << 'END'
+1..6
+ok 1
+ok 2
+ok 3
+END
+
+cat > bot << 'END'
+ok 5
+ok 6
+END
+
+cat top - bot > skip.test << 'END'
+ok # SKIP
+::skip::
+END
+
+cat top - bot > todo.test << 'END'
+not ok # TODO
+::todo::
+END
+
+cat top - bot > fail.test << 'END'
+not ok
+::fail::
+END
+
+cat top - bot > xpass.test << 'END'
+ok # TODO
+::xpass::
+END
+
+cat top - bot > bail.test << 'END'
+::bail::
+Bail out!
+END
+
+cat top - bot > error.test << 'END'
+::error::
+1..7
+END
+
+# Created with "dd if=/dev/urandom count=1 | base64-encode"
+cat > hodgepodge <<'END'
+1+0 records in
+1+0 records out
+512 bytes (512 B) copied, 0.000241092 s, 2.1 MB/s
+gdivnV4VhL4DOzhE3zULJuun3PwqqQqMdATVcZbIQkNgyRvNBoqqHMBQEs7QsjDbp2nK+Szz
+EcelGyvXmHrW7yImaom6Yrg95k31VCmp/pGDRnTDwDrcOPJiv9jDReEmTAQkPuqLO+mFNly+
+DDHM9fNbzGumstsQ3wq3DOXz1pCV3JXwhjeaHSboeEbmr55bX0XHLSKaecctA0rXDXEyZWZ/
+ODlawSrAXzw0H7a+xBwjnNXZ3zYiwk3x+WQrPqNjb+qXiLLTxAKzx2/KnaFhxkPlte5jPRNB
+FciDolL+H/10LsCdSzLOrGnY2zH6vL2JMZfxjnb73zWFcdWWE01LTD7wpN5O1MP3+N47lcVe
+lWbkD04cJvhwxLElYSO24B743GG5EyGYt9SeZRE6xbgwq3fVOS8KqjwGxwi4adSBTtw0CV8W
+S/6n8dck1vBvjA+qpk0zMSYSqc3+jzW9UiGTmTEIwfw80p/lGNsfjcNBJ86nFkWUnHmrsi8k
+Dv57sK70mTg239g08f5Uvdga/5UreMBSgB0hUj5sbq57r7B1fsVr0Kag468la8zKy3ZEZ0gX
+++sbaJ9WGHhnKvjooeH+4Y6HwAFsdINde++FlCmp4EuNKKEEuXbSKLaOTy3+6pJ2DYdvRCL+
+frZwxH4hcrw8qh+8IakB02viewZS/qT57v4=
+END
+
+exec 5>misc.test
+echo 'ok # SKIP' >&5
+cat hodgepodge >&5
+echo 'not ok # TODO' >&5
+echo 'ok' >&5
+exec 5>&-
+
+cat > skipall.test << 'END'
+1..0 # SKIP all
+END
+
+# We don't care about the exit status of "make check" here.
+TESTS="`echo *.test`" $MAKE -e check || :
+cat test-suite.log
+
+grep ':.*ok|not seen' test-suite.log && Exit 1
+
+for s in skip todo fail xpass bail error; do
+ $FGREP "::$s::" test-suite.log
+done
+
+grep '^1\.\.0 # SKIP all$' test-suite.log
+
+case `cat test-suite.log` in
+ *"`cat hodgepodge`"*) ;;
+ *) Exit 1;;
+esac
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - which global test result derives from different test results
+# mixed in a single script?
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > ok.test <<END
+1..3
+ok 1
+not ok 2 # TODO
+ok 3 # SKIP
+END
+
+cat > skip.test <<'END'
+1..3
+ok 1 # SKIP
+ok 2 # SKIP
+ok 3 # SKIP
+END
+
+cat > skipall.test <<'END'
+1..0 # SKIP
+foo
+# bar
+END
+
+cat > fail.test <<'END'
+1..1
+not ok 1
+END
+
+(sed '1s/.*/1..4/' ok.test && echo 'not ok 4') > fail2.test
+
+cat > xpass.test <<'END'
+1..1
+ok 1 # TODO
+END
+
+(sed '1s/.*/1..4/' ok.test && echo 'ok 4 # TODO') > xpass2.test
+
+echo 'Bail out!' > bail.test
+
+(cat ok.test && echo 'Bail out!') > bail2.test
+
+cat > bail3.test <<'END'
+1..0 # SKIP
+Bail out!
+END
+
+# Too many tests.
+cat > error.test <<'END'
+1..2
+ok 1
+ok 2 # SKIP
+not ok 3
+not ok 4 # TODO
+END
+
+# Too few tests.
+cat > error2.test <<'END'
+1..4
+ok 1
+not ok 2 # TODO
+ok 3 # SKIP
+END
+
+# Repeated plan.
+cat > error3.test <<'END'
+1..2
+1..2
+ok 1
+ok 2
+END
+
+# Too many tests, after a "SKIP" plan.
+cat > error4.test <<'END'
+1..0 # SKIP
+ok 1
+ok 2
+END
+
+# Tests out of order.
+cat > error5.test <<'END'
+1..4
+not ok 1 # TODO
+ok 3
+ok 2
+ok 4
+END
+
+# Wrong test number.
+cat > error6.test <<'END'
+1..2
+ok 1 # SKIP
+ok 7
+END
+
+# No plan.
+cat > error7.test <<'END'
+ok 1 # SKIP
+ok 2 # TODO
+not ok 3 # TODO
+ok 4
+END
+
+cat > hodgepodge.test <<'END'
+1..2
+not ok 1
+ok 2 # TODO
+Bail out!
+END
+
+cat > hodgepodge-all.test <<'END'
+1..4
+ok 1
+ok 2 # SKIP
+not ok 2 # TODO
+not ok 3
+ok 4 # TODO
+Bail out!
+END
+
+tests=`echo *.test` # Also required later.
+
+TESTS="$tests" $MAKE -e check >stdout && { cat stdout; Exit 1; }
+cat stdout
+
+# Dirty trick required here.
+for tst in `echo " $tests " | sed 's/.test / /'`; do
+ echo :copy-in-global-log: yes >> $tst.trs
+done
+
+rm -f test-suite.log
+TESTS="$tests" $MAKE -e test-suite.log && Exit 1
+cat test-suite.log
+
+have_rst_section ()
+{
+ eqeq=`echo "$1" | sed 's/./=/g'`
+ # Assume $1 contains no RE metacharacters.
+ sed -n "/^$1$/,/^$eqeq$/p" test-suite.log > got
+ (echo "$1" && echo "$eqeq") > exp
+ cat exp
+ cat got
+ diff exp got
+}
+
+have_rst_section 'PASS: ok'
+have_rst_section 'SKIP: skip'
+have_rst_section 'SKIP: skipall'
+have_rst_section 'FAIL: fail'
+have_rst_section 'FAIL: fail2'
+have_rst_section 'FAIL: xpass'
+have_rst_section 'FAIL: xpass2'
+have_rst_section 'ERROR: bail'
+have_rst_section 'ERROR: bail2'
+have_rst_section 'ERROR: bail3'
+have_rst_section 'ERROR: error'
+have_rst_section 'ERROR: error2'
+have_rst_section 'ERROR: error3'
+have_rst_section 'ERROR: error4'
+have_rst_section 'ERROR: error5'
+have_rst_section 'ERROR: error6'
+have_rst_section 'ERROR: error7'
+have_rst_section 'ERROR: hodgepodge'
+have_rst_section 'ERROR: hodgepodge-all'
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - "check-html" and "recheck-html" targets
+# - reStructuredText -> HTML conversion
+
+parallel_tests=yes
+required=rst2html
+. ./defs || Exit 1
+
+cat > Makefile.am << 'END'
+TEST_LOG_COMPILER = cat
+TESTS = ok.test ko.test
+END
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > ok.test << 'END'
+1..3
+ok 1 - expected success
+not ok 2 - expectred failure # TODO
+ok 3 - skipped test # SKIP
+END
+
+cat > ko.test << 'END'
+1..2
+not ok 1 - unexpected failure
+ok 2 - spurious success # TODO
+Bail out! hard error
+END
+
+no_rst2html_error ()
+{
+ $EGREP '(test-suite\.log:[0-9]|WARNING|SEVERE)' output && Exit 1
+ :
+}
+
+# Overriding TESTS should work with check-html.
+TESTS=ok.test $MAKE -e check-html >output 2>&1 || { cat output; Exit 1; }
+cat output
+no_rst2html_error
+test -f test-suite.html
+
+mv output stdout
+count_test_results total=3 pass=1 fail=0 xpass=0 xfail=1 skip=1 error=0
+
+: > older
+$sleep
+$MAKE check-html >output 2>&1 && { cat output; Exit 1; }
+cat output
+no_rst2html_error
+# test-suite.html should be remake if if it was up-to-date.
+test -f test-suite.html
+is_newest test-suite.html older
+
+mv output stdout
+count_test_results total=6 pass=1 fail=1 xpass=1 xfail=1 skip=1 error=1
+
+rm -f ok.test
+$MAKE recheck-html >output 2>&1 && { cat output; Exit 1; }
+cat output
+no_rst2html_error
+
+mv output stdout
+count_test_results total=3 pass=0 fail=1 xpass=1 xfail=0 skip=0 error=1
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - log file creation
+# - log file removal
+# - stdout and stderr of a script go in its log file
+# - TEST_SUITE_LOG redefinition, at either automake or make time
+# - VERBOSE environment variable support
+# Keep in sync with 'test-log.test'.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat > Makefile.am << 'END'
+TESTS = pass.test skip.test xfail.test fail.test xpass.test error.test
+TEST_SUITE_LOG = global.log
+END
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+# Custom markers, for use in grepping checks.
+cmarker=::: # comment marker
+pmarker=%%% # plain maker
+
+cat > pass.test <<END
+#! /bin/sh -e
+echo 1..1
+echo "$pmarker pass $pmarker" >&2
+echo "# $cmarker pass $cmarker" >&2
+echo "ok 1"
+END
+
+cat > skip.test <<END
+#! /bin/sh -e
+echo 1..1
+echo "$pmarker skip $pmarker"
+echo "# $cmarker skip $cmarker"
+echo "ok 1 # SKIP"
+END
+
+cat > xfail.test <<END
+#! /bin/sh -e
+echo 1..1
+echo "$pmarker xfail $pmarker" >&2
+echo "# $cmarker xfail $cmarker" >&2
+echo "not ok 1 # TODO"
+END
+
+cat > fail.test <<END
+#! /bin/sh -e
+echo 1..1
+echo "$pmarker fail $pmarker"
+echo "# $cmarker fail $cmarker"
+echo "not ok 1"
+END
+
+cat > xpass.test <<END
+#! /bin/sh -e
+echo 1..1
+echo "$pmarker xpass $pmarker" >&2
+echo "# $cmarker xpass $cmarker" >&2
+echo "ok 1 # TODO"
+END
+
+cat > error.test <<END
+#! /bin/sh -e
+echo 1..1
+echo "$pmarker error $pmarker"
+echo "# $cmarker error $cmarker"
+echo 'Bail out!'
+END
+
+chmod a+x *.test
+
+TEST_SUITE_LOG=my.log $MAKE -e check && Exit 1
+ls -l # For debugging.
+test ! -f test-suite.log
+test ! -f global.log
+test -f my.log
+st=0
+for result in pass fail xfail xpass skip error; do
+ cat $result.log # For debugging.
+ $FGREP "$pmarker $result $pmarker" $result.log || st=1
+ $FGREP "$cmarker $result $cmarker" $result.log || st=1
+done
+test $st -eq 0 || Exit 1
+cat my.log # For debugging.
+for result in xfail fail xpass skip error; do
+ cat $result.log # For debugging.
+ $FGREP "$pmarker $result $pmarker" my.log || st=1
+ $FGREP "$cmarker $result $cmarker" my.log || st=1
+done
+test `$FGREP -c "$pmarker" my.log` -eq 5
+test `$FGREP -c "$cmarker" my.log` -eq 5
+
+# Passed test scripts shouldn't be mentioned in the global log.
+$EGREP '(^pass|[^x]pass)\.test' my.log && Exit 1
+# But failing (expectedly or not) and skipped ones should.
+$FGREP 'xfail.test' my.log
+$FGREP 'skip.test' my.log
+$FGREP 'fail.test' my.log
+$FGREP 'xpass.test' my.log
+$FGREP 'error.test' my.log
+
+touch error2.log test-suite.log global.log
+TEST_SUITE_LOG=my.log $MAKE -e mostlyclean
+ls -l # For debugging.
+test ! -f my.log
+test ! -f pass.log
+test ! -f fail.log
+test ! -f xfail.log
+test ! -f xpass.log
+test ! -f skip.log
+test ! -f error.log
+# "make mostlyclean" shouldn't remove unrelated log files.
+test -f error2.log
+test -f test-suite.log
+test -f global.log
+
+rm -f *.log
+
+VERBOSE=yes $MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+cat global.log
+test ! -f my.log
+test ! -f test-suite.log
+# Check that VERBOSE causes the global testsuite log to be
+# emitted on stdout.
+out=`cat stdout`
+log=`cat global.log`
+case $out in *"$log"*) ;; *) Exit 1;; esac
+
+touch error2.log test-suite.log my.log
+$MAKE clean
+ls -l # For debugging.
+test ! -f global.log
+test ! -f pass.log
+test ! -f fail.log
+test ! -f xfail.log
+test ! -f xpass.log
+test ! -f skip.log
+test ! -f error.log
+# "make clean" shouldn't remove unrelated log files.
+test -f error2.log
+test -f test-suite.log
+test -f my.log
+
+rm -f *.log
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - The Automake TAP driver has an option that instruct it to read TAP
+# input also from the stderr of the test command, not only its stdout.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+fetch_tap_driver
+
+cat > Makefile.am << 'END'
+AM_TEST_LOG_DRIVER_FLAGS = --comments --merge
+TESTS = all.test
+END
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > all.test <<END
+#!/bin/sh
+echo 1..4
+echo ok 1 >&2
+echo ok 2
+echo "not ok 3 # TODO" >&2
+echo "ok 4 # SKIP"
+echo "# foo foo foo" >&2
+END
+chmod a+x all.test
+
+$MAKE check >stdout || { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=4 pass=2 fail=0 xpass=0 xfail=1 skip=1 error=0
+grep '^# all\.test: foo foo foo' stdout
+
+cat > all.test <<END
+#!/bin/sh
+echo 1..1
+echo ok 1
+echo 'Bail out!' >&2
+END
+
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=2 pass=1 fail=0 xpass=0 xfail=0 skip=0 error=1
+
+# See that the option `--no-merge' can override the effect of `--merge'.
+
+TEST_LOG_DRIVER_FLAGS=--no-merge $MAKE -e check >stdout \
+ || { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=1 pass=1 fail=0 xpass=0 xfail=0 skip=0 error=0
+
+
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - if a test script exits with non-zero status before printing the TAP
+# plan, then the driver reports both "missing plan" and "exited with
+# non-zero status" errors.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat > foo.test <<END
+#!/bin/sh
+exit 123
+END
+
+echo TESTS = foo.test > Makefile.am
+
+chmod a+x foo.test
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+count_test_results total=2 pass=0 fail=0 xpass=0 xfail=0 skip=0 error=2
+
+grep '^ERROR: foo\.test - exited with status 123$' stdout
+grep '^ERROR: foo\.test - missing test plan$' stdout
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# More on TAP support:
+# - more than one TAP-generating test script in $(TESTS)
+# - VPATH builds
+# - use with parallel make (if supported)
+# - basic use of diagnostic messages (lines beginning with "#")
+# - flags for TAP driver defined through AC_SUBST in configure.ac
+# - messages generated by the testsuite harness reference the
+# correct test script(s)
+# - "make distcheck" works
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+fetch_tap_driver
+
+cat >> configure.in <<END
+AC_SUBST([AM_TEST_LOG_DRIVER_FLAGS], ['--comments'])
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+TEST_LOG_DRIVER = $(srcdir)/tap-driver
+TESTS = 1.test 2.test 3.test
+EXTRA_DIST = $(TESTS) tap-driver
+END
+
+cat > 1.test <<'END'
+#! /bin/sh
+echo 1..2
+echo ok 1 - mu
+if test -f not-skip; then
+ echo "not ok 2 zardoz"
+else
+ echo "ok 2 zardoz # SKIP"
+fi
+END
+
+cat > 2.test <<'END'
+#! /bin/sh
+echo 1..3
+echo "ok"
+echo "not ok # TODO not implemented"
+echo "ok 3"
+END
+
+cat > 3.test <<END
+#! /bin/sh
+echo 1..1
+echo ok - blah blah blah
+echo '# Some diagnostic'
+if test -f bail-out; then
+ echo 'Bail out! Kernel Panic'
+else
+ :
+fi
+END
+
+chmod a+x [123].test
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE
+
+# Try a VPATH and by default serial build first, and then an in-tree
+# and by default parallel build.
+for try in 0 1; do
+
+ if test $try -eq 0; then
+ # VPATH serial build.
+ mkdir build
+ cd build
+ srcdir=..
+ run_make=$MAKE
+ elif test $try -eq 1; then
+ # In-tree parallel build.
+ srcdir=.
+ case $MAKE in
+ *\ -j*)
+ # Degree of parallelism already specified by the user: do
+ # not override it.
+ run_make=$MAKE
+ ;;
+ *)
+ # Some make implementations (e.g., HP-UX) don't grok `-j',
+ # some require no space between `-j' and the number of jobs
+ # (e.g., older GNU make versions), and some *do* require a
+ # space between `-j' and the number of jobs (e.g., Solaris
+ # dmake). We need a runtime test to see what works.
+ echo 'all:' > Makefile
+ for run_make in "$MAKE -j3" "$MAKE -j 3" "$MAKE"; do
+ $run_make && break
+ done
+ rm -f Makefile
+ ;;
+ esac
+ else
+ fatal_ "internal error, invalid value of '$try' for \$try"
+ fi
+
+ $srcdir/configure
+ ls -l # For debugging.
+
+ # Success.
+
+ $run_make check >stdout || { cat stdout; Exit 1; }
+ cat stdout
+ count_test_results total=6 pass=4 fail=0 xpass=0 xfail=1 skip=1 error=0
+ grep '^PASS: 1\.test 1 - mu$' stdout
+ grep '^SKIP: 1\.test 2 zardoz # SKIP$' stdout
+ test `$FGREP -c '1.test' stdout` -eq 2
+ grep '^PASS: 2\.test 1$' stdout
+ grep '^XFAIL: 2\.test 2 # TODO not implemented$' stdout
+ grep '^PASS: 2\.test 3$' stdout
+ test `$FGREP -c '2.test' stdout` -eq 3
+ grep '^PASS: 3\.test 1 - blah blah blah$' stdout
+ grep '^# 3\.test: Some diagnostic$' stdout
+ test `$FGREP -c '3.test' stdout` -eq 2
+
+ # Failure.
+
+ # Use `echo' here, since Solaris 10 /bin/sh would try to optimize
+ # a `:' away after the first iteration, even if it is redirected.
+ echo dummy > not-skip
+ echo dummy > bail-out
+ $run_make check >stdout && { cat stdout; Exit 1; }
+ cat stdout
+ count_test_results total=7 pass=4 fail=1 xpass=0 xfail=1 skip=0 error=1
+ grep '^PASS: 1\.test 1 - mu$' stdout
+ grep '^FAIL: 1\.test 2 zardoz$' stdout
+ test `$FGREP -c '1.test' stdout` -eq 2
+ grep '^PASS: 2\.test 1$' stdout
+ grep '^XFAIL: 2\.test 2 # TODO not implemented$' stdout
+ grep '^PASS: 2\.test 3$' stdout
+ test `$FGREP -c '2.test' stdout` -eq 3
+ grep '^PASS: 3\.test 1 - blah blah blah$' stdout
+ grep '^# 3\.test: Some diagnostic$' stdout
+ grep '^ERROR: 3\.test - Bail out! Kernel Panic$' stdout
+ test `$FGREP -c '3.test' stdout` -eq 3
+
+ cd $srcdir
+
+done
+
+$MAKE distcheck
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# More on TAP support:
+# - more LOG_COMPILER at once for TAP tests
+# - binary programs in $(TESTS)
+# - interactions with `check_*' variables
+
+required='cc native'
+parallel_tests=yes
+. ./defs || Exit 1
+
+fetch_tap_driver
+
+cat >> configure.in <<END
+AC_PROG_CC
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+TEST_EXTENSIONS = .sh .tap
+
+tap_driver = $(srcdir)/tap-driver
+
+LOG_DRIVER = $(tap_driver)
+SH_LOG_DRIVER = $(tap_driver)
+TAP_LOG_DRIVER = $(tap_driver)
+
+TAP_LOG_COMPILER = cat
+SH_LOG_COMPILER = $(SHELL)
+LOG_COMPILER =
+
+EXTRA_DIST = baz.tap
+
+check_SCRIPTS = bar.sh
+bar.sh: Makefile
+ echo '#!/bin/sh' > $@-t
+ echo 'echo 1..1' >> $@-t
+ echo 'echo "not ok 1 # TODO"' >> $@-t
+ chmod a-w $@-t && mv -f $@-t $@
+CLEANFILES = bar.sh
+
+check_PROGRAMS = foo-test
+foo_test_SOURCES = foo.c
+
+TESTS = foo-test $(check_SCRIPTS) baz.tap
+
+EXTRA_DIST += tap-driver
+END
+
+cat > foo.c <<'END'
+#include <stdio.h>
+int main (void)
+{
+ printf ("1..1\n");
+ printf ("ok 1\n");
+ return 0;
+}
+END
+
+cat > baz.tap << 'END'
+1..1
+ok 1 # SKIP
+END
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE
+
+./configure
+
+for target in check distcheck; do
+ $MAKE $target >stdout || { cat stdout; Exit 1; }
+ cat stdout
+ count_test_results total=3 pass=1 fail=0 xpass=0 xfail=1 skip=1 error=0
+done
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - literal strings "0" and "0.0" as a test description in the message
+# of a "plan with skip" TAP plan
+# generally true!
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+echo 'Bail out! 0' > a.test
+echo 'Bail out! 0.0' > b.test
+
+TESTS='a.test b.test' $MAKE -e check >stdout && { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=2 pass=0 fail=0 xpass=0 xfail=0 skip=0 error=2
+
+grep '^ERROR: a.test - Bail out! 0$' stdout
+grep '^ERROR: b.test - Bail out! 0\.0$' stdout
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - literal string "0" as a TODO or SKIP message
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > all.test << 'END'
+1..3
+ok 1 # TODO 0
+not ok 2 # TODO 0
+ok 3 # SKIP 0
+END
+
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=3 pass=0 fail=0 xpass=1 xfail=1 skip=1 error=0
+
+grep '^XPASS: all\.test 1 # TODO 0$' stdout
+grep '^XFAIL: all\.test 2 # TODO 0$' stdout
+grep '^SKIP: all\.test 3 # SKIP 0$' stdout
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - literal "0" and "0.0" in a test description and a TODO/SKIP message
+# at the same time
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > all.test << 'END'
+1..14
+
+ok 1 0
+ok 2 0.0
+
+ok 3 0 # TODO 0
+ok 4 0.0 # TODO 0
+ok 5 0 # TODO 0.0
+ok 6 0.0 # TODO 0.0
+
+not ok 7 0 # TODO 0
+not ok 8 0.0 # TODO 0
+not ok 9 0 # TODO 0.0
+not ok 10 0.0 # TODO 0.0
+
+ok 11 0 # SKIP 0
+ok 12 0.0 # SKIP 0
+ok 13 0 # SKIP 0.0
+ok 14 0.0 # SKIP 0.0
+
+END
+
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=14 pass=2 fail=0 xpass=4 xfail=4 skip=4 error=0
+
+sed '/^ *$/d' > exp << 'END'
+
+PASS: all.test 1 0
+PASS: all.test 2 0.0
+
+XPASS: all.test 3 0 # TODO 0
+XPASS: all.test 4 0.0 # TODO 0
+XPASS: all.test 5 0 # TODO 0.0
+XPASS: all.test 6 0.0 # TODO 0.0
+
+XFAIL: all.test 7 0 # TODO 0
+XFAIL: all.test 8 0.0 # TODO 0
+XFAIL: all.test 9 0 # TODO 0.0
+XFAIL: all.test 10 0.0 # TODO 0.0
+
+SKIP: all.test 11 0 # SKIP 0
+SKIP: all.test 12 0.0 # SKIP 0
+SKIP: all.test 13 0 # SKIP 0.0
+SKIP: all.test 14 0.0 # SKIP 0.0
+
+END
+
+$FGREP ': all.test' stdout > got
+
+cat exp
+cat got
+diff exp got
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - literal strings "0" and "0.0" as the reason of the skip in a "TAP
+# plan with skip" (i.e., "1..0 # SKIP ...").
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+echo '1..0 # SKIP 0' > a.test
+echo '1..0 # SKIP 0.0' > b.test
+
+TESTS='a.test b.test' $MAKE -e check >stdout || { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=2 pass=0 fail=0 xpass=0 xfail=0 skip=2 error=0
+
+grep '^SKIP: a.test - 0$' stdout
+grep '^SKIP: b.test - 0\.0$' stdout
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - the string "0" as a test description
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > all.test << 'END'
+1..10
+ok 1 0
+ok - 0
+not ok 3 0
+not ok - 0
+ok 5 0 # TODO
+ok - 0 # TODO
+not ok 7 0 # TODO
+not ok - 0 # TODO
+ok 9 0 # SKIP
+ok - 0 # SKIP
+END
+
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=10 pass=2 fail=2 xpass=2 xfail=2 skip=2 error=0
+
+cat > exp << 'END'
+PASS: all.test 1 0
+PASS: all.test 2 - 0
+FAIL: all.test 3 0
+FAIL: all.test 4 - 0
+XPASS: all.test 5 0 # TODO
+XPASS: all.test 6 - 0 # TODO
+XFAIL: all.test 7 0 # TODO
+XFAIL: all.test 8 - 0 # TODO
+SKIP: all.test 9 0 # SKIP
+SKIP: all.test 10 - 0 # SKIP
+END
+
+$FGREP ': all.test' stdout > got
+
+cat exp
+cat got
+diff exp got
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - don't spuriously recognize negative TAP result numbers, but correctly
+# interpret them as test descriptions instead
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > all.test <<'END'
+1..7
+ok -1
+not ok -3
+ok -2 # SKIP
+not ok -5 # TODO
+ok -04 # TODO
+ok -121
+not ok -50000
+END
+
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=7 pass=2 fail=2 xpass=1 xfail=1 skip=1 error=0
+
+cat > exp <<'END'
+PASS: all.test 1 -1
+FAIL: all.test 2 -3
+SKIP: all.test 3 -2 # SKIP
+XFAIL: all.test 4 -5 # TODO
+XPASS: all.test 5 -04 # TODO
+PASS: all.test 6 -121
+FAIL: all.test 7 -50000
+END
+
+$FGREP ': all.test' stdout > got
+
+cat exp
+cat got
+diff exp got
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - "Bail out!" magic and TAP parse errors are not disabled nor turned
+# into simple failures by the definition DISABLE_HARD_ERRORS.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat > Makefile.am << 'END'
+DISABLE_HARD_ERRORS = yes
+TEST_LOG_COMPILER = cat
+TESTS = bail.test few.test noplan.test
+END
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > bail.test <<END
+1..1
+Bail out!
+END
+
+cat > few.test <<END
+1..1
+END
+
+cat > noplan.test <<END
+# nothing here
+END
+
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=3 pass=0 fail=0 xpass=0 xfail=0 skip=0 error=3
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - By default, TAP input is only from the stdout (and not the stderr)
+# of the test command.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat > Makefile.am << 'END'
+TEST_LOG_DRIVER_FLAGS = --comments
+TESTS = all.test
+END
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > all.test <<END
+#!/bin/sh
+echo 1..2
+echo 'not ok' >&2
+echo 'not ok 1 # TODO' >&2
+echo 'ok 1'
+echo '# foo foo foo'
+echo '# bar bar bar' >&2
+echo 'ok 44 # TODO' >&2
+echo 'Bail out!' >&2
+echo 'ok 2 # SKIP'
+END
+
+chmod a+x all.test
+
+$MAKE check >stdout || { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=2 pass=1 fail=0 xpass=0 xfail=0 skip=1 error=0
+$FGREP 'foo foo foo' stdout
+$FGREP 'bar bar bar' stdout && Exit 1
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - we shouldn't spuriously recognize as TAP result numbers what it
+# not, even if it seems pretty close
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > prefixes <<'END'
+A
+a
+_
++
+-
+=
+/
+*
+.
+:
+,
+;
+$
+@
+%
+&
+#
+?
+!
+|
+\
+"
+`
+'
+(
+)
+[
+]
+{
+}
+<
+>
+END
+
+n=`wc -l <prefixes`
+
+# See the loop below to understand this initialization.
+pass=`expr $n '*' 3`
+fail=$pass
+skip=`expr $pass - 3`
+xfail=$skip
+xpass=$xfail
+error=0
+total=`expr $pass + $fail + $skip + $xfail + $xpass`
+
+echo 1..$total > all.test
+
+highno=1000
+
+for result in 'ok' 'not ok'; do
+ for spacing in "" " " "$tab"; do
+ subst="$result &$spacing$higno"
+ sed -e "s|.*|$subst|" prefixes
+ for directive in TODO SKIP; do
+ test "$result $directive" != "not ok SKIP" || continue
+ sed -e '/^#$/d' -e "s|.*|$subst # $directive|" prefixes
+ done
+ done
+done >> all.test
+
+cat all.test # For debugging.
+
+# Sanity checks.
+grep '#.*#' all.test && framework_failure_ "creating all.test"
+test `wc -l <all.test` -lt $highno || framework_failure_ "creating all.test"
+
+$MAKE check >stdout || :
+cat stdout
+
+count_test_results total=$total pass=$pass fail=$fail skip=$skip \
+ xpass=$xpass xfail=$xfail error=$error
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - `:test-results:' directives in test scripts' output doesn't
+# originate spurious results in the testsuite summary
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > all.test <<'END'
+1..1
+:test-result: PASS
+:test-result: FAIL
+:test-result: XPASS
+:test-result: XFAIL
+:test-result: SKIP
+:test-result: ERROR
+:test-result: UNKNOWN
+ok 1
+END
+
+$MAKE check >stdout || { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=1 pass=1 fail=0 xpass=0 xfail=0 skip=0 error=0
+
+for result in PASS FAIL XPASS XFAIL SKIP ERROR UNKNOWN; do
+ grep "^ *:test-result: $result$" all.log
+done
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - don't spuriously recognize lines that are "almost" TAP lines as
+# real TAP lines
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+echo 1..5 > all.test
+
+# The only recognized directives are "TODO" and "SKIP".
+# So the following should count as passed tests.
+cat >> all.test <<END
+ok 1 # XFAIL
+ok 2 # SKIPPED
+ok 3 # TO DO
+ok 4 # TODOALL
+ok 5 # FIXME
+END
+
+# According to documentation of Test::Harness::TAP(3):
+#
+# ``Lines written to standard output matching /^(not )?ok\b/
+# must be interpreted as test lines. All other lines must
+# not be considered test output.''
+
+cat >> all.test <<END
+ ok
+ ok 1
+${tab}ok
+${tab}ok 1
+ not ok
+ not ok 1
+${tab}not ok
+${tab}not ok 1
+notok
+notok 1
+not${tab}ok
+not${tab}ok 1
+not ok
+not ok 1
+no ok
+no ok 1
+# ok
+# not ok
+# ok 1
+# not ok 1
+#ok
+#not ok
+#ok 1
+#not ok 1
+END
+
+set +x # Don't pollute logs too much.
+for r in 'ok' 'not ok'; do
+ for s1 in \
+ a b c d e f g h i j k l m n o p q r s t u v w x y z \
+ A B C D E F G H I J K L M N O P Q R S T U V W X Y Z \
+ 0 1 2 3 4 5 6 7 8 9 _ ab 0a 23 a1B2c _o _x_y_
+ do
+ for s2 in '' @ % + - = / . : \; \* \? \& \! \# \$ \< \> \\; do
+ printf '%s\n' "$r$s1$s2"
+ done
+ done
+done >> all.test
+set -x # Reset shell xtraces.
+
+# The prove(1) utility doesn't bail out on these, so our driver
+# shouldn't either.
+cat >> all.test <<'END'
+bailout
+bailout!
+bail out
+bail out!
+Bailout
+Bailout!
+Bail out
+ Bail out
+ Bail out!
+#Bail out!
+# Bail out!
+END
+
+cat all.test # For debugging.
+
+# Minor sanity check.
+test `grep -c '^ok1$' all.test` -eq 1 \
+ && test `grep -c '^not ok1$' all.test` -eq 1 \
+ || framework_failure_ "creating all.test"
+
+$MAKE check >stdout || { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=5 pass=5 fail=0 xpass=0 xfail=0 skip=0 error=0
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - a "not ok # SKIP" line should count as a failure, for consistency
+# with the prove(1) utility.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > all.test <<'END'
+1..4
+not ok # SKIP
+not ok 2 # SKIP
+not ok - foo # SKIP
+not ok 4 - bar # SKIP
+END
+
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=4 pass=0 fail=4 skip=0 xpass=0 xfail=0 error=0
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - TAP result numbers terminated by a non-whitespace "word boundary"
+# character are recognized
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > all.test <<'END'
+1..5
+ok 1: foo1
+not ok 2- foo2
+ok 3@ foo3 # SKIP
+not ok 4?&%$ foo4 # TODO
+ok 5"`!! foo5 # TODO
+END
+
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=5 pass=1 fail=1 skip=1 xpass=1 xfail=1 error=0
+
+$EGREP '^PASS: all\.test 1 ?: foo1$' stdout
+$EGREP '^FAIL: all\.test 2 ?- foo2$' stdout
+$EGREP '^SKIP: all\.test 3 ?@ foo3 # SKIP$' stdout
+$EGREP '^XFAIL: all\.test 4 ?\?&%\$ foo4 # TODO$' stdout
+$EGREP '^XPASS: all\.test 5 ?"`!! foo5 # TODO$' stdout
+
+cat > all.test <<'END'
+1..3
+ok 2: quux
+not ok 1!
+ok 476$ # SKIP
+END
+
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=3 pass=0 fail=0 skip=0 xpass=0 xfail=0 error=3
+
+$EGREP '^ERROR: all\.test 2 ?: quux # OUT-OF-ORDER \(expecting 1\)$' stdout
+$EGREP '^ERROR: all\.test 1 ?! # OUT-OF-ORDER \(expecting 2\)$' stdout
+$EGREP '^ERROR: all\.test 476 ?\$ # OUT-OF-ORDER \(expecting 3\)$' stdout
+
+cat > punctuation <<'END'
+'
+"
+`
+#
+$
+!
+\
+/
+&
+%
+(
+)
+|
+^
+~
+?
+*
++
+-
+,
+:
+;
+=
+<
+>
+@
+[
+]
+{
+}
+\
+END
+
+# Strip any extra whitespace, for Solaris' wc.
+planned=`wc -l <punctuation | tr -d " $tab"`
+
+echo 1..$planned > all.test
+awk '{print "ok " NR $0 }' punctuation >> all.test
+# For debugging.
+cat all.test
+# Sanity check.
+grep "^ok 1'$" all.test || framework_failure_ "creating all.test"
+
+$MAKE check >stdout || { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=$planned pass=$planned \
+ fail=0 skip=0 xpass=0 xfail=0 error=0
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - how does TAP result numbers with leading zero fares?
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+do_checks ()
+{
+ $MAKE check >stdout && { cat stdout; Exit 1; }
+ cat stdout
+ count_test_results "$@"
+ # Allow some normalization of leading zeros, without forcing it.
+ LC_ALL=C sed -n 's/: all\.test 0*\([0-9]\)/: all.test \1/p' stdout > got
+ cat exp
+ cat got
+ diff exp got
+}
+
+cat > all.test <<END
+1..15
+
+ok 01
+ok 0002
+
+not ok 03
+not ok 0004
+
+ok 05 # SKIP
+ok 0006 # SKIP
+
+not ok 07 # TODO
+not ok 0008 # TODO
+
+ok 009
+ok 010
+
+ok 00000011
+not ok 0012
+ok 00000013 # SKIP
+not ok 0014 # TODO
+ok 00000015 # TODO
+END
+
+cat > exp <<END
+PASS: all.test 1
+PASS: all.test 2
+FAIL: all.test 3
+FAIL: all.test 4
+SKIP: all.test 5 # SKIP
+SKIP: all.test 6 # SKIP
+XFAIL: all.test 7 # TODO
+XFAIL: all.test 8 # TODO
+PASS: all.test 9
+PASS: all.test 10
+PASS: all.test 11
+FAIL: all.test 12
+SKIP: all.test 13 # SKIP
+XFAIL: all.test 14 # TODO
+XPASS: all.test 15 # TODO
+END
+
+do_checks total=15 pass=5 fail=3 xpass=1 xfail=3 skip=3 error=0
+
+cat > all.test <<END
+1..8
+
+ok 010
+not ok 003
+ok 0001 # SKIP
+not ok 010 # TODO
+ok 00100 # TODO
+
+ok 06
+ok 00007
+ok 8
+END
+
+cat > exp <<END
+ERROR: all.test 10 # OUT-OF-ORDER (expecting 1)
+ERROR: all.test 3 # OUT-OF-ORDER (expecting 2)
+ERROR: all.test 1 # OUT-OF-ORDER (expecting 3)
+ERROR: all.test 10 # OUT-OF-ORDER (expecting 4)
+ERROR: all.test 100 # OUT-OF-ORDER (expecting 5)
+PASS: all.test 6
+PASS: all.test 7
+PASS: all.test 8
+END
+
+do_checks total=8 pass=3 fail=0 xpass=0 xfail=0 skip=0 error=5
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP result lines whose description is a number.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+# Some random numbers to be used as test names. The definitions below are
+# selected so that $x<n> != <n> for every n >= 1. We can't use positional
+# parameters for this unfortunately, since Solaris 10 /bin/sh doesn't allow
+# the dereferencing of parameters from ${10} onwards.
+x1=5 x2=7 x3=45 x4=11 x5=7 x6=3 x7=6 x8=9 x9=1000 x10=1
+
+cat > all.test <<END
+1..10
+ok 1 ${x1}
+ok - ${x2}
+not ok 3 ${x3}
+not ok - ${x4}
+ok 5 ${x5} # SKIP
+ok - ${x6} # SKIP
+not ok 7 ${x7} # TODO
+not ok - ${x8} # TODO
+ok 9 ${x9} # TODO
+ok - ${x10} # TODO
+END
+
+cat > exp <<END
+PASS: all.test 1 ${x1}
+PASS: all.test 2 - ${x2}
+FAIL: all.test 3 ${x3}
+FAIL: all.test 4 - ${x4}
+SKIP: all.test 5 ${x5} # SKIP
+SKIP: all.test 6 - ${x6} # SKIP
+XFAIL: all.test 7 ${x7} # TODO
+XFAIL: all.test 8 - ${x8} # TODO
+XPASS: all.test 9 ${x9} # TODO
+XPASS: all.test 10 - ${x10} # TODO
+END
+
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=10 pass=2 fail=2 xpass=2 xfail=2 skip=2 error=0
+
+$FGREP ': all.test' stdout > got
+
+cat exp
+cat got
+diff exp got
+
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - out-of-order test results
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > a.test <<END
+1..3
+ok
+not ok 2 # TODO
+ok 71
+END
+
+cat > b.test <<END
+1..5
+ok 1
+ok 2
+ok 4 foo # SKIP mu
+not ok 2 bar # TODO um
+ok 5
+END
+
+cat > c.test <<END
+1..4
+ok 1
+ok foo
+ok 4 - zardoz
+not ok 3
+END
+
+# This is documented as an "test out of sequence" error in the
+# TAP::Parser documentation. Keep it in sync with the example
+# there.
+cat > d.test <<'END'
+1..5
+ok 1
+ok 2
+ok 15
+ok 16
+ok 17
+END
+
+TESTS='a.test b.test c.test d.test' $MAKE -e check >stdout \
+ && { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=17 pass=8 fail=0 xpass=0 xfail=1 skip=0 error=8
+
+test `$FGREP -c ': a.test' stdout` -eq 3
+test `$FGREP -c ': b.test' stdout` -eq 5
+test `$FGREP -c ': c.test' stdout` -eq 4
+test `$FGREP -c ': d.test' stdout` -eq 5
+
+grep '^ERROR: a\.test 71 # OUT-OF-ORDER (expecting 3)$' stdout
+grep '^ERROR: b\.test 4 foo # OUT-OF-ORDER (expecting 3)$' stdout
+grep '^ERROR: b\.test 2 bar # OUT-OF-ORDER (expecting 4)$' stdout
+grep '^ERROR: c\.test 4 - zardoz # OUT-OF-ORDER (expecting 3)$' stdout
+grep '^ERROR: c\.test 3 # OUT-OF-ORDER (expecting 4)$' stdout
+grep '^ERROR: d\.test 15 # OUT-OF-ORDER (expecting 3)$' stdout
+grep '^ERROR: d\.test 16 # OUT-OF-ORDER (expecting 4)$' stdout
+grep '^ERROR: d\.test 17 # OUT-OF-ORDER (expecting 5)$' stdout
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - non-success exit status of a test script is reported in the
+# log file
+# See also related test 'tap-passthrough.test'.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat > Makefile.am << 'END'
+TEST_LOG_COMPILER = $(SHELL)
+TESTS =
+END
+
+exit_statuses='1 2 77 99'
+
+for e in $exit_statuses; do
+ unindent > exit-$e.test <<END
+ #!/bin/sh
+ echo 1..1
+ echo ok 1
+ exit $e
+END
+ echo TESTS += exit-$e.test >> Makefile.am
+done
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+st=0
+$MAKE check || st=$?
+for e in $exit_statuses; do cat exit-$e.log; done
+cat test-suite.log
+test $st -gt 0 || Exit 1
+
+for e in $exit_statuses; do
+ for log in exit-$e.log test-suite.log; do
+ grep "^ERROR: exit-$e\\.test - exited with status $e$" $log
+ done
+done
+
+env TEST_LOG_DRIVER_FLAGS='--ignore-exit' $MAKE -e check
+$FGREP ".test - exited with status" *.log && Exit 1
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - all input (valid TAP lines, invalid TAP lines, non-TAP lines)
+# are passed through in the log file
+# - TAP errors are reported in the log file too
+# See also related test 'tap-passthrough-exit.test'.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+weirdchars=\''"\$@!&()[]<>#;,:.^?*/'
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+#
+# Only successful tests.
+#
+
+# The whitespace in this test might be normalized in the testsuite
+# progress output, but should be copied verbatim in the log files.
+cat > ok.test <<END
+1..6
+TAP plan in the previous line.
+ok${tab}
+ok 2
+ok - foo
+ok 4 - x
+ This is not a TAP line, but should still be copied in the log file!
+# some diagnostic${tab}
+not ok # TODO low priority
+ok # SKIP who cares?
+$weirdchars
+END
+
+TESTS=ok.test $MAKE -e check || { cat ok.log; Exit 1; }
+cat ok.log
+
+for rx in \
+ '1\.\.6' \
+ 'TAP plan in the previous line\.' \
+ "ok${tab}" \
+ 'ok 2' \
+ 'ok - foo' \
+ 'ok 4 - x' \
+ ' This is not a TAP line, but should still be copied in the log file!' \
+ "# some diagnostic${tab}" \
+ 'not ok # TODO low priority' \
+ 'ok # SKIP who cares?' \
+; do
+ grep "^$rx$" ok.log
+done
+$FGREP "$weirdchars" ok.log
+
+#
+# Mixed failing/successful tests.
+#
+
+cat > tiny.test <<END
+1..1
+ok
+END
+
+cat > ok.test <<END
+1..1
+ok
+only one success here
+END
+
+cat > ko.test <<END
+1..5
+foo foo foo
+ok${tab}
+ok 2
+not ok - foo
+not ok 4 - x
+# diagnostic ko
+ bar${tab}bar${tab}bar
+ok # TODO dunno
+$weirdchars
+END
+
+cat > bail.test <<END
+Bail out! Test is taking too long!
+END
+
+cat > skip.test <<END
+1..0 # Skipped: WWW::Mechanize not installed
+END
+
+cat > err.test <<END
+1..3
+ok 1
+Invalid test count
+ok 23
+Misplaced plan
+1..13
+ok
+Extra test
+ok
+Last line
+END
+
+st=0
+env TESTS='tiny.test ok.test ko.test bail.test skip.test err.test' \
+ $MAKE -e check || st=$?
+cat tiny.log
+cat ok.log
+cat ko.log
+cat bail.log
+cat skip.log
+cat err.log
+test $st -gt 0 || Exit 1
+
+grep '^1\.\.1$' tiny.log
+grep '^ok$' tiny.log
+grep '^only one success here$' ok.log
+
+for rx in \
+ '1\.\.5' \
+ 'foo foo foo' \
+ "ok${tab}" \
+ 'ok 2' \
+ 'not ok - foo' \
+ 'not ok 4 - x' \
+ '# diagnostic ko' \
+ " bar${tab}bar${tab}bar" \
+ 'ok # TODO dunno' \
+; do
+ grep "^$rx$" ko.log
+done
+$FGREP "$weirdchars" ko.log
+
+grep '^Bail out! Test is taking too long!$' bail.log
+grep '^1\.\.0 # Skipped: WWW::Mechanize not installed$' skip.log
+
+for rx in \
+ '^1\.\.3$' \
+ '^Invalid test count$' \
+ '^ok 23$' \
+ '^Misplaced plan$' \
+ '^1\.\.13$' \
+ '^ERROR:.* multiple test plans' \
+ '^Extra test$' \
+ '^Last line$' \
+ '^ERROR:.* [tT]oo many tests run.*expected 3, got 4' \
+ '^ERROR:.* err\.test 23 .*OUT[ -]OF[ -]ORDER.*expecting 2' \
+; do
+ grep "$rx" err.log
+done
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - some corner cases for TAP plan
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+# -------------------------------------------------------------------------
+
+cat > leading-repeated.test <<END
+1..1
+1..1
+ok 1
+END
+
+cat > trailing-repeated.test <<END
+ok 1
+1..1
+1..1
+END
+
+for pos in leading trailing; do
+ TESTS="$pos-repeated.test" $MAKE -e check >stdout \
+ && { cat stdout; Exit 1; }
+ cat stdout
+ count_test_results total=2 pass=1 fail=0 xpass=0 xfail=0 skip=0 error=1
+ grep "^ERROR: $pos-repeated\\.test - multiple test plans$" stdout
+done
+
+# -------------------------------------------------------------------------
+
+cat > leading-repeated.test <<END
+1..2
+ok 1
+1..2
+ok 2
+END
+
+cat > trailing-repeated.test <<END
+ok 1
+1..2
+ok 2
+1..2
+END
+
+env TESTS="leading-repeated.test trailing-repeated.test" \
+ $MAKE -e check >stdout && { cat stdout; Exit 1; }
+cat stdout
+count_test_results total=6 pass=3 fail=0 xpass=0 xfail=0 skip=0 error=3
+grep "^ERROR: leading-repeated\\.test - multiple test plans$" stdout
+grep "^ERROR: trailing-repeated\\.test - multiple test plans$" stdout
+grep "^ERROR: trailing-repeated\\.test 2 # AFTER LATE PLAN$" stdout
+grep "leading .*AFTER LATE PLAN" stdout && Exit 1
+
+# -------------------------------------------------------------------------
+
+cat > 1.test <<END
+1..0
+1..0
+END
+
+cat > 2.test <<END
+1..0 # SKIP
+1..0
+END
+
+cat > 3.test <<END
+1..0
+1..0 # SKIP
+END
+
+cat > 4.test <<END
+1..0 # SKIP
+1..0 # SKIP
+END
+
+env TESTS="1.test 2.test 3.test 4.test" \
+ $MAKE -e check >stdout && { cat stdout; Exit 1; }
+cat stdout
+count_test_results total=8 pass=0 fail=0 xpass=0 xfail=0 skip=4 error=4
+for i in 1 2 3 4; do
+ grep "^ERROR: $i\\.test - multiple test plans$" stdout
+done
+
+# -------------------------------------------------------------------------
+
+cat > all.test <<END
+1..5
+ok 1
+ok 2
+1..5
+ok 3
+1..5
+ok 4
+1..5
+ok 5
+END
+
+$MAKE -e check >stdout && { cat stdout; Exit 1; }
+cat stdout
+count_test_results total=8 pass=5 fail=0 xpass=0 xfail=0 skip=0 error=3
+
+cat > exp <<'END'
+PASS: all.test 1
+PASS: all.test 2
+ERROR: all.test - multiple test plans
+PASS: all.test 3
+ERROR: all.test - multiple test plans
+PASS: all.test 4
+ERROR: all.test - multiple test plans
+PASS: all.test 5
+END
+
+$FGREP ': all.test' stdout > got
+
+cat exp
+cat got
+diff exp got
+
+# -------------------------------------------------------------------------
+
+cat > all.test <<END
+1..2
+ok 1
+ok 2
+1..3
+ok 3
+END
+
+$MAKE -e check >stdout && { cat stdout; Exit 1; }
+cat stdout
+count_test_results total=5 pass=2 fail=0 xpass=0 xfail=0 skip=0 error=3
+
+cat > exp <<'END'
+PASS: all.test 1
+PASS: all.test 2
+ERROR: all.test - multiple test plans
+ERROR: all.test 3 # UNPLANNED
+ERROR: all.test - too many tests run (expected 2, got 3)
+END
+
+$FGREP ': all.test' stdout > got
+
+cat exp
+cat got
+diff exp got
+
+# -------------------------------------------------------------------------
+
+cat > all.test <<END
+ok 1
+ok 2
+ok 3
+1..5
+1..5
+ok 4
+1..5
+ok 5
+END
+
+$MAKE -e check >stdout && { cat stdout; Exit 1; }
+cat stdout
+count_test_results total=7 pass=3 fail=0 xpass=0 xfail=0 skip=0 error=4
+
+cat > exp <<'END'
+PASS: all.test 1
+PASS: all.test 2
+PASS: all.test 3
+ERROR: all.test - multiple test plans
+ERROR: all.test 4 # AFTER LATE PLAN
+ERROR: all.test - multiple test plans
+ERROR: all.test 5 # AFTER LATE PLAN
+END
+
+$FGREP ': all.test' stdout > got
+
+cat exp
+cat got
+diff exp got
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support: the following situations should be flagged as errors:
+# - unmatched test plan (too few tests run)
+# - multiple test plans
+# - missing test plan
+# - misplaced test plan (tests run after a late plan)
+# Checks about unplanned tests are performed in 'tap-unplanned.test'
+# and 'tap-skip-whole-badcount.test'. More checks about corner-cases
+# in TAP plans are performed in 'tap-plan-corner.test'.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+my_check ()
+{
+ cat > all.test
+ test -n "$err" || fatal_ "\$err not set before calling my_check"
+ cat all.test # For debugging.
+ $MAKE check >stdout && { cat stdout; Exit 1; }
+ cat stdout
+ count_test_results "$@"
+ grep "^ERROR: all\\.test $err$" stdout
+ unset err
+}
+
+err='- too few tests run (expected 2, got 1)'
+my_check total=2 pass=1 fail=0 xpass=0 xfail=0 skip=0 error=1 <<END
+1..2
+ok 1
+END
+
+err='- too few tests run (expected 12, got 3)'
+my_check total=4 pass=2 fail=0 xpass=0 xfail=1 skip=0 error=1 <<END
+ok 1
+ok 2
+not ok 3 # TODO
+1..12
+END
+
+err='- too few tests run (expected 1, got 0)'
+my_check total=1 pass=0 fail=0 xpass=0 xfail=0 skip=0 error=1 <<END
+1..1
+END
+
+err='2 # AFTER LATE PLAN'
+my_check total=2 pass=1 fail=0 xpass=0 xfail=0 skip=0 error=1 <<END
+ok 1
+1..2
+ok 2
+END
+
+err='5 # AFTER LATE PLAN'
+my_check total=5 pass=4 fail=0 xpass=0 xfail=0 skip=0 error=1 <<END
+ok 1
+ok 2
+ok 3
+ok 4
+1..5
+ok 5
+END
+
+err='- missing test plan'
+my_check total=2 pass=1 fail=0 xpass=0 xfail=0 skip=0 error=1 <<END
+ok 1
+END
+
+
+# The two test plans here are deliberately equal.
+err='- multiple test plans'
+my_check total=3 pass=2 fail=0 xpass=0 xfail=0 skip=0 error=1 <<END
+1..2
+ok 1
+ok 2
+1..2
+END
+
+# The second plan is diagnosed as extra, and only the first one is
+# relevant w.r.t. the number of the expected test.
+err='- multiple test plans'
+my_check total=4 pass=3 fail=0 xpass=0 xfail=0 skip=0 error=1 <<END
+1..3
+ok 1
+ok 2
+1..2
+ok 3
+END
+
+# As above, in a slightly different situation.
+err='- multiple test plans'
+my_check total=3 pass=2 fail=0 xpass=0 xfail=0 skip=0 error=1 <<END
+1..2
+ok 1
+ok 2
+1..4
+END
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - TAP plans with numbers having leading zeroes, as in "1..01"
+# - TAP "SKIP" plans with multiple zeroes, as in "1..00 # SKIP"
+# This is consistent with the behaviour of the `prove' utility.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > a.test <<END
+1..01
+ok 1
+END
+
+cat > b.test <<END
+1..0002
+ok 1
+ok 2
+END
+
+echo 1..010 > c.test
+for i in 1 2 3 4 5 6 7 8 9 10; do echo ok $i; done >> c.test
+
+echo 1..00100 > d.test
+for i in 1 2 3 4 5 6 7 8 9 10; do
+ for j in 1 2 3 4 5 6 7 8 9 10; do
+ echo ok
+ done
+done >> d.test
+
+echo 1..00 > e.test
+echo '1..000 # SKIP' > f.test
+
+env TESTS='a.test b.test c.test d.test e.test f.test' \
+ $MAKE -e check >stdout || { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=115 pass=113 xfail=0 skip=2 fail=0 xpass=0 error=0
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support: a malformed TAP plan is not recognized. The checks in
+# here should be consistent with the behaviour of the `prove' utility.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > a.test <<END
+1..1 foo
+END
+
+cat > b.test <<END
+ 1..2
+ok 1
+ok 2
+END
+
+cat > c.test <<END
+1..1 # SKIP
+END
+
+cat > d.test <<END
+1..2 # foo bar
+not ok 1 # TODO
+not ok 2 # TODO
+END
+
+cat > e.test <<END
+0..0
+END
+
+tests_list=`echo *.test`
+
+TESTS="$tests_list" $MAKE -e check >stdout && { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=9 pass=2 fail=0 xpass=0 xfail=2 skip=0 error=5
+
+for tst in $tests_list; do
+ grep "^ERROR: $tst - missing test plan$" stdout
+done
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - test plan preceding and/or following non-result TAP lines
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > top1.test <<END
+non-TAP line, ignored
+1..1
+ok 1
+END
+
+cat > top2.test <<END
+some
+non-TAP
+lines
+are
+ignored
+# and a TAP comment won't cause problems either
+1..2
+ok 1
+ok 2
+END
+
+# Try with a blank line too, just to be sure.
+cat > top3.test <<END
+
+1..1
+ok 1
+END
+
+cat > bot1.test <<END
+ok 1 # SKIP
+1..1
+bla blah blah ...
+END
+
+cat > bot2.test <<END
+ok 1
+ok 2
+not ok 3 # TODO
+1..3
+#@$%! (a cursing comment :-)
+END
+
+# Try with a blank line too, just to be sure.
+cat > bot3.test <<END
+ok 1
+not ok 2 # TODO
+ok 3 # SKIP
+ok 4 # SKIP
+1..4
+
+END
+
+tests=`echo *.test`
+
+for tap_flags in "" "--comments"; do
+ env TEST_LOG_DRIVER_FLAGS="$tap_flags" TESTS="$tests" \
+ $MAKE -e check >stdout || { cat stdout; Exit 1; }
+ cat stdout
+ count_test_results total=12 pass=7 xfail=2 skip=3 fail=0 xpass=0 error=0
+done
+
+:
#! /bin/sh
-# Copyright (C) 2001, 2002, 2010, 2011 Free Software Foundation, Inc.
+# Copyright (C) 2011 Free Software Foundation, Inc.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Test for bug reported by Eric Magnien.
+# TAP support:
+# - plan line with trailing whitespace is recognized and handled correctly
+parallel_tests=yes
. ./defs || Exit 1
-cat > configure.in <<END
-AC_INIT([$me], [1.0])
-AM_INIT_AUTOMAKE
-AC_OUTPUT([subdir/Makefile subdir/foo Makefile foo], [true])
-END
-
-mkdir subdir
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
-: > Makefile.am
-: > subdir/Makefile.am
-: > foo.in
-: > subdir/foo.in
+sed 's/\$$//' > foo.test <<END
+1..2 $
+ok 1$
+ok 2$
+END
-$ACLOCAL
-$AUTOMAKE
+cat > bar.test <<END
+1..1$tab $tab$tab
+ok 1
+END
-count=`$FGREP foo.in Makefile.in | wc -l`
-test $count -eq 2
+TESTS='foo.test bar.test' $MAKE -e check >stdout || { cat stdout; Exit 1; }
+cat stdout
-# This ought to work as well.
-$ACLOCAL
-$AUTOMAKE --add-missing --force-missing
+count_test_results total=3 pass=3 fail=0 error=0 xpass=0 xfail=0 skip=0
:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - test scripts with the test plan at the beginning
+# - test scripts with the test plan at the end
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > top.test <<END
+1..3
+ok 1
+ok 2 # SKIP
+ok 3
+# a trailing comment
+END
+
+cat > bot.test <<END
+ok 1
+# a comment
+ok 2
+not ok 3 # TODO
+# another comment
+ok
+1..4
+END
+
+for tap_flags in "" "--comments"; do
+ env TEST_LOG_DRIVER_FLAGS="$tap_flags" TESTS='top.test bot.test' \
+ $MAKE -e check >stdout || { cat stdout; Exit 1; }
+ cat stdout
+ count_test_results total=7 pass=5 xfail=1 skip=1 fail=0 xpass=0 error=0
+done
+
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - interactions between "TAP plan with SKIP" and logging of earlier or
+# later TAP or non-TAP text
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+echo TEST_LOG_DRIVER_FLAGS = --comments >> Makefile
+
+cat > foo.test <<END
+1..0
+a non-TAP line
+# a comment
+END
+
+cat > foo2.test <<END
+a non-TAP line 2
+# a comment 2
+1..0
+END
+
+cat > bar.test <<END
+# an early comment
+an early non-TAP line
+ $tab
+1..0 # SKIP
+# a later comment
+a later non-TAP line
+END
+
+TESTS='foo.test foo2.test bar.test' $MAKE -e check >stdout \
+ || { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=3 pass=0 fail=0 xpass=0 xfail=0 skip=3 error=0
+
+grep '^# foo\.test: a comment$' stdout
+grep '^# foo2\.test: a comment 2$' stdout
+grep '^# bar\.test: an early comment$' stdout
+grep '^# bar\.test: a later comment$' stdout
+
+cat foo.log
+cat foo2.log
+cat bar.log
+
+grep '^a non-TAP line$' foo.log
+grep '^# a comment$' foo.log
+grep '^a non-TAP line 2$' foo2.log
+grep '^# a comment 2$' foo2.log
+grep '^# an early comment' bar.log
+grep '^an early non-TAP line$' bar.log
+grep '^# a later comment' bar.log
+grep '^a later non-TAP line$' bar.log
+grep "^ $tab$" bar.log
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - an exit status != 0 of a test script causes an hard error, even if
+# the last line of output is a "SKIP plan" (e.g., "1..0 # SKIP").
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+echo TESTS = one.test two.test > Makefile.am
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > one.test <<'END'
+#!/bin/sh
+echo '1..0 # SKIP'
+exit 1
+END
+
+cat > two.test <<'END'
+#!/bin/sh
+echo '1..0'
+exit 22
+END
+
+chmod a+x one.test two.test
+
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+
+# The 'prove' utility reports both the skip and the non-zero exit status,
+# so we do the same.
+count_test_results total=4 pass=0 fail=0 xpass=0 xfail=0 skip=2 error=2
+
+grep '^ERROR: one\.test - exited with status 1$' stdout
+grep '^ERROR: two\.test - exited with status 22$' stdout
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - a "Bail out!" directive causes an hard error, even if coming after
+# a "SKIP plan" (e.g., "1..0 # SKIP").
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > all.test <<END
+1..0 # SKIP
+Bail out!
+END
+
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+count_test_results total=2 pass=0 fail=0 xpass=0 xfail=0 skip=1 error=1
+grep '^ERROR: all\.test - Bail out!' stdout
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - "SKIP" keyword in a TAP plan is case-insensitive
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+# These nested loops are clearer without indentation.
+
+i=0
+for c1 in s S; do
+for c2 in k K; do
+for c3 in i I; do
+for c4 in p P; do
+ i=`expr $i + 1`
+ case $i in ?) i=0$i; esac
+ echo "1..0 # $c1$c2$c3$c4 foobar" > $i.test
+done; done; done; done
+
+TESTS="`echo *.test`" $MAKE -e check >stdout || { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=16 pass=0 fail=0 xpass=0 xfail=0 skip=16 error=0
+
+for tst in *.test; do
+ grep "^SKIP: $tst .* foobar$" stdout
+done
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - the special "plan with SKIP" can also be used "late" in the TAP
+# stream, i.e., preceded by non-TAP output or TAP diagnostic.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > foo.test <<END
+some non-TAP text, will be copied in the global log
+1..0 # SKIP from the last line
+END
+
+cat > bar.test <<END
+# some TAP diagnostic, will go to console
+1..0
+END
+
+TESTS='foo.test bar.test' $MAKE -e check >stdout || { cat stdout; Exit 1; }
+cat stdout
+
+grep '^SKIP: foo\.test .* from the last line$' stdout
+grep '^SKIP: bar\.test$' stdout
+test `grep -c ': .*\.test' stdout` -eq 2
+count_test_results total=2 pass=0 fail=0 xpass=0 xfail=0 skip=2 error=0
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - TAP errors following a "TAP plan with SKIP" are still diagnosed.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > all.test <<END
+1..0
+a non-TAP line
+1..3
+1..5
+# a comment
+1..1
+END
+
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=4 pass=0 fail=0 xpass=0 xfail=0 skip=1 error=3
+test `grep -c '^ERROR: all\.test - multiple test plans' stdout` -eq 3
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - test results seen in a TAP stream that has a "plan with SKIP" are
+# flagged as errors, even if all of them are "SKIP" results.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > foo.test <<END
+1..0 # SKIP
+ok 1 # SKIP
+END
+
+cat > bar.test <<END
+ok 1 # SKIP
+1..0 # SKIP
+END
+
+TESTS='foo.test bar.test' $MAKE -e check >stdout && { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=5 pass=0 fail=0 xpass=0 xfail=0 skip=2 error=3
+
+# Don't be too strict w.r.t. the format of the "skip whole test" message.
+grep '^SKIP: foo\.test$' stdout || grep '^SKIP: foo\.test [^0-9]' stdout
+grep '^ERROR: foo\.test 1 # UNPLANNED$' stdout
+grep '^ERROR: foo\.test - too many tests run (expected 0, got 1)$' stdout
+grep '^SKIP: bar\.test 1 # SKIP$' stdout
+grep '^ERROR: bar\.test - too many tests run (expected 0, got 1)$' stdout
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - a "plan with skip" given after one or more test result have already
+# been seen is an error
+# - any test result following a "plan with skip" is an error.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > all.test <<END
+ok 1
+1..0 # SKIP too late
+END
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+count_test_results total=2 pass=1 fail=0 xpass=0 xfail=0 skip=0 error=1
+grep '^ERROR: all\.test - too many tests run (expected 0, got 1)$' stdout
+
+cat > all.test <<END
+ok 1
+ok 2 # SKIP
+not ok 3 # TODO
+1..0 # SKIP too late
+END
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+count_test_results total=4 pass=1 fail=0 xpass=0 xfail=1 skip=1 error=1
+grep '^ERROR: all\.test - too many tests run (expected 0, got 3)$' stdout
+
+cat > all.test <<END
+1..0 # SKIP falsified later
+ok 1
+END
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+count_test_results total=3 pass=0 fail=0 xpass=0 xfail=0 skip=1 error=2
+grep '^ERROR: all\.test 1 # UNPLANNED$' stdout
+grep '^ERROR: all\.test - too many tests run (expected 0, got 1)$' stdout
+
+cat > all.test <<END
+1..0 # SKIP falsified later
+ok 1
+ok 2 # SKIP
+not ok 3
+not ok 4 # TODO
+END
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+count_test_results total=6 pass=0 fail=0 xpass=0 xfail=0 skip=1 error=5
+grep '^ERROR: all\.test 1 # UNPLANNED$' stdout
+grep '^ERROR: all\.test 2 # UNPLANNED$' stdout
+grep '^ERROR: all\.test 3 # UNPLANNED$' stdout
+grep '^ERROR: all\.test 4 # UNPLANNED$' stdout
+grep '^ERROR: all\.test - too many tests run (expected 0, got 4)$' stdout
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - normalization of whitespace in console testsuite progress associated
+# with a SKIP directive in the TAP plan
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > foo.test <<END
+1..0${tab}${tab} #${tab}SKIP ${tab}Strip leading & trailing ${tab}${tab}
+END
+
+cat > bar.test <<END
+1..0 #SKIP Preserve ${tab} middle${tab}${tab}space
+END
+
+cat > baz.test <<END
+1..0 # SKIP${tab} Strip${tab}external preserve ${tab}middle ${tab}${sp}
+END
+
+TESTS='foo.test bar.test baz.test' $MAKE -e check > stdout \
+ || { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=3 pass=0 fail=0 error=0 xpass=0 xfail=0 skip=3
+
+grep "SKIP: foo\\.test - Strip leading & trailing$" stdout
+grep "SKIP: bar\\.test - Preserve ${tab} middle${tab}${tab}space" stdout
+grep "SKIP: baz\\.test - Strip${tab}external preserve ${tab}middle" stdout
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Basic TAP test protocol support:
+# - special plan format to skip all the tests in a script
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+weirdchars=\''"$!&()[]<>#;^?*/@%=,.:'
+
+cat > foo.test <<END
+1..0
+END
+
+cat > bar.test <<END
+blah
+# blah
+1..0$tab $tab
+END
+
+# It is undefined whether the comment after the plan below should
+# count as an explanation; still, the test should be skipped.
+cat > baz.test <<END
+1..0 # WWW::Mechanize not installed
+other
+ junk
+ lines
+END
+
+cat > wget.test <<END
+1..0 # SKIP wget(1) not installed
+# See also curl.test
+END
+
+cat > curl.test <<END
+1..0 # SKIP: Can't connect to gnu.org!
+# See also wget.test
+END
+
+cat > mu.test <<END
+1..0 # SKIP $weirdchars
+END
+
+env TESTS='foo.test bar.test baz.test wget.test curl.test mu.test' \
+ $MAKE -e check >stdout || { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=6 pass=0 fail=0 xpass=0 xfail=0 skip=6 error=0
+
+# Look for a regression where the "1..0" wasn't being stripped from the
+# SKIP message.
+$FGREP '1..0' stdout && Exit 1
+
+grep '^SKIP: foo\.test$' stdout
+grep '^SKIP: bar\.test$' stdout
+grep '^SKIP: baz\.test' stdout # Deliberately laxer, see above for why.
+grep '^SKIP: wget\.test .* wget(1) not installed$' stdout
+grep '^SKIP: curl\.test .* Can'\''t connect to gnu\.org!$' stdout
+grep '^SKIP: mu\.test' stdout | $FGREP "$weirdchars" stdout
+test `grep -c ': .*\.test' stdout` -eq 6
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - testsuite progress on console should happen mostly "in real time";
+# i.e., it's not acceptable for the driver to wait the end of the
+# script to start displaying results from it.
+# FIXME: this test uses expect(1) to ensure line buffering from make and
+# children, and is pretty hacky and complex; is there a better way to
+# accomplish the checks done here?
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >expect-check <<'END'
+eval spawn $env(SHELL) -c ":"
+expect eof
+END
+expect -f expect-check || {
+ echo "$me: failed to find a working expect program" >&2
+ Exit 77
+}
+rm -f expect-check
+
+cat > Makefile.am << 'END'
+TESTS = all.test
+END
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > all.test <<'END'
+#! /bin/sh
+echo 1..3
+
+# FIXME: creative quoting to placate maintainer-check
+sleep="sleep "3
+
+# The awk+shell implementation of the TAP driver must "read ahead" of one
+# line in order to catch the exit status of the test script it runs. So
+# be sure to echo one "dummy" line after each result line in order not to
+# cause false positives.
+
+echo ok 1 - foo
+echo DUMMY
+$sleep
+test -f ok-1 || { echo 'Bail out!'; exit 1; }
+
+echo ok 2 - bar
+echo DUMMY
+$sleep
+test -f ok-2 || { echo 'Bail out!'; exit 1; }
+
+echo ok 3 - baz
+echo DUMMY
+$sleep
+test -f ok-3 || { echo 'Bail out!'; exit 1; }
+
+: > all-is-well
+END
+
+chmod a+x all.test
+
+cat > expect-make <<'END'
+eval spawn $env(MAKE) check
+expect {
+ "PASS: all.test 1 - foo" {
+ open "ok-1" "w"
+ exp_continue
+ }
+ "PASS: all.test 2 - bar" {
+ open "ok-2" "w"
+ exp_continue
+ }
+ "PASS: all.test 3 - baz" {
+ open "ok-3" "w"
+ exp_continue
+ }
+ "Testsuite summary" {
+ exit 0
+ }
+ timeout {
+ puts "expect timed out"
+ exit 1
+ }
+ default {
+ puts "expect error"
+ exit 1
+ }
+}
+END
+
+# Expect should simulate a tty as stdout, which should ensure a
+# line-buffered output.
+MAKE=$MAKE expect -f expect-make
+test -f all-is-well
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - RECHECK_LOGS
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat > Makefile.am << 'END'
+TEST_LOG_COMPILER = cat
+TESTS = foo.test bar.test baz.test
+baz.log: zardoz
+END
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+: > zardoz
+
+cat > foo.test <<'END'
+1..2
+ok 1
+ok 2
+END
+
+cat > bar.test <<'END'
+1..1
+not ok 1
+END
+
+cat > baz.test <<'END'
+1..1
+Bail out!
+END
+
+# Even the tests that are not re-run should contribute to the testsuite
+# summary when obtained by "make check RECHECK_LOGS=".
+grep_summary ()
+{
+ grep '^# TOTAL: *4$' stdout
+ grep '^# PASS: *2$' stdout
+ grep '^# XPASS: *0$' stdout
+ grep '^# FAIL: *1$' stdout
+ grep '^# XFAIL: *0$' stdout
+ grep '^# SKIP: *0$' stdout
+ grep '^# ERROR: *1$' stdout
+}
+
+$MAKE -e check && Exit 1
+test -f foo.log
+test -f bar.log
+test -f baz.log
+
+rm -f foo.log bar.log
+
+env RECHECK_LOGS= $MAKE -e check > stdout && { cat stdout; Exit 1; }
+cat stdout
+test -f foo.log
+test -f bar.log
+grep '^PASS: foo\.test 1$' stdout
+grep '^PASS: foo\.test 2$' stdout
+grep '^FAIL: bar\.test 1$' stdout
+grep 'baz\.test' stdout && Exit 1
+grep_summary
+
+$sleep
+touch foo.test
+# We re-run only a successful test, but the tests that failed in the
+# previous run should still be taken into account, and cause an overall
+# failure.
+env RECHECK_LOGS= $MAKE -e check > stdout && { cat stdout; Exit 1; }
+cat stdout
+grep '^PASS: foo\.test 1$' stdout
+grep '^PASS: foo\.test 2$' stdout
+grep 'ba[rz]\.test' stdout && Exit 1
+is_newest foo.log foo.test
+grep_summary
+
+$sleep
+touch zardoz
+env RECHECK_LOGS= $MAKE -e check > stdout && { cat stdout; Exit 1; }
+cat stdout
+grep '^ERROR: baz\.test' stdout
+$EGREP '(foo|bar)\.test' stdout && Exit 1
+is_newest baz.log zardoz
+grep_summary
+
+# Now, explicitly retry with all test logs already updated, and ensure
+# that the summary is still displayed.
+env RECHECK_LOGS= $MAKE -e check > stdout && { cat stdout; Exit 1; }
+cat stdout
+$EGREP '(foo|bar|baz)\.test' stdout && Exit 1
+grep_summary
+
+# The following should re-run foo.test (and only foo.test), even if its
+# log file is up-to-date.
+: > older
+env RECHECK_LOGS=foo.log $MAKE -e check > stdout && { cat stdout; Exit 1; }
+cat stdout
+grep '^PASS: foo\.test 1$' stdout
+grep '^PASS: foo\.test 2$' stdout
+grep 'ba[rz]\.test' stdout && Exit 1
+is_newest foo.log older
+grep_summary
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Test the 'recheck' target for TAP test protocol.
+# Keep in sync with 'test-driver-custom-multitest-recheck.test'.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+fetch_tap_driver
+
+cat >> configure.in <<END
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+TEST_LOG_DRIVER = $(srcdir)/tap-driver
+TESTS = a.test b.test c.test d.test
+END
+
+cat > a.test << 'END'
+#! /bin/sh
+echo 1..2
+echo ok 1
+echo ok 2
+: > a.run
+END
+
+cat > b.test << 'END'
+#! /bin/sh
+: > b.run
+echo 1..2
+echo ok 1
+if test -f b.ok; then
+ echo ok 2
+else
+ echo 'Bail out!'
+fi
+END
+
+cat > c.test << 'END'
+#! /bin/sh
+echo 1..3
+if test -f c.pass; then
+ echo 'ok - c is ok :-)'
+else
+ echo 'not ok - c is ko :-('
+fi
+if test -f c.xfail; then
+ echo 'not ok 2 # TODO'
+else
+ echo 'ok 2 # TODO'
+fi
+echo 'not ok 3 - blah blah # TODO need better diagnostic'
+: > c.run
+END
+
+cat > d.test << 'END'
+#! /bin/sh
+test -f ./d.count && . ./d.count
+echo 1..${test_count-2}
+echo ok 1 '# SKIP: who cares ...'
+(. ./d.extra) || echo 'not ok 2 - d.extra failed'
+: > d.run
+END
+
+chmod a+x *.test
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE
+
+do_recheck ()
+{
+ case $* in
+ --fail) on_bad_rc='&&';;
+ --pass) on_bad_rc='||';;
+ *) fatal_ "invalid usage of function 'do_recheck'";;
+ esac
+ rm -f *.run
+ eval "\$MAKE recheck >stdout $on_bad_rc { cat stdout; ls -l; Exit 1; }; :"
+ cat stdout; ls -l
+}
+
+for vpath in : false; do
+ if $vpath; then
+ mkdir build
+ cd build
+ srcdir=..
+ else
+ srcdir=.
+ fi
+
+ $srcdir/configure
+
+ : A "make recheck" in a clean tree should run no tests.
+ do_recheck --pass
+ cat test-suite.log
+ test ! -r a.run
+ test ! -r a.log
+ test ! -r b.run
+ test ! -r b.log
+ test ! -r c.run
+ test ! -r c.log
+ test ! -r d.run
+ test ! -r d.log
+ count_test_results total=0 pass=0 fail=0 xpass=0 xfail=0 skip=0 error=0
+
+ : Run the tests for the first time.
+ $MAKE check >stdout && { cat stdout; Exit 1; }
+ cat stdout
+ ls -l
+ # All the test scripts should have run.
+ test -f a.run
+ test -f b.run
+ test -f c.run
+ test -f d.run
+ count_test_results total=9 pass=3 fail=2 xpass=1 xfail=1 skip=1 error=1
+
+ : Let us make b.test pass.
+ echo OK > b.ok
+ do_recheck --fail
+ # a.test has been successful the first time, so no need to re-run it.
+ # Similar considerations apply to similar checks, below.
+ test ! -r a.run
+ test -f b.run
+ test -f c.run
+ test -f d.run
+ count_test_results total=7 pass=2 fail=2 xpass=1 xfail=1 skip=1 error=0
+
+ : Let us make the first part of c.test pass.
+ echo OK > c.pass
+ do_recheck --fail
+ test ! -r a.run
+ test ! -r b.run
+ test -f c.run
+ test -f d.run
+ count_test_results total=5 pass=1 fail=1 xpass=1 xfail=1 skip=1 error=0
+
+ : Let us make also the second part of c.test pass.
+ echo KO > c.xfail
+ do_recheck --fail
+ test ! -r a.run
+ test ! -r b.run
+ test -f c.run
+ test -f d.run
+ count_test_results total=5 pass=1 fail=1 xpass=0 xfail=2 skip=1 error=0
+
+ : Nothing changed, so only d.test should be run.
+ for i in 1 2; do
+ do_recheck --fail
+ test ! -r a.run
+ test ! -r b.run
+ test ! -r c.run
+ test -f d.run
+ count_test_results total=2 pass=0 fail=1 xpass=0 xfail=0 skip=1 error=0
+ done
+
+ : Let us make d.test run more testcases, and experience _more_ failures.
+ echo 'test_count=9' > d.count
+ unindent > d.extra <<'END'
+ echo 'ok # SKIP s'
+ echo 'not ok - f 1'
+ echo 'ok - p 1'
+ echo 'not ok - f 2'
+ echo 'ok # TODO xp'
+ echo 'not ok - f 3'
+ echo 'not ok - f 4'
+ echo 'ok - p 2'
+ echo 'ok' # Extra test.
+ echo 'Bail out!'
+END
+ do_recheck --fail
+ test ! -r a.run
+ test ! -r b.run
+ test ! -r c.run
+ test -f d.run
+ count_test_results total=11 pass=2 fail=4 xpass=1 xfail=0 skip=2 error=2
+
+ : Let us finally make d.test pass.
+ echo 'test_count=1' > d.count
+ echo : > d.extra
+ do_recheck --pass
+ test ! -r a.run
+ test ! -r b.run
+ test ! -r c.run
+ test -f d.run
+ count_test_results total=1 pass=0 fail=0 xpass=0 xfail=0 skip=1 error=0
+
+ : All tests have been successful or skipped, nothing should be re-run.
+ do_recheck --pass
+ test ! -r a.run
+ test ! -r b.run
+ test ! -r c.run
+ test ! -r d.run
+ count_test_results total=0 pass=0 fail=0 xpass=0 xfail=0 skip=0 error=0
+
+ cd $srcdir
+
+done
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - non-directive comments in TAP results are kept verbatim
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > all.test <<END
+1..5
+ok 1 # TDO
+not ok 2 # SKP
+ok #SIKP${tab}${sp}
+not ok # TOD${tab}
+ok 5 # ${tab}${tab}TOOD${tab}${sp}${sp}
+END
+
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=5 pass=3 fail=2 xpass=0 xfail=0 skip=0 error=0
+
+# Don't be too strict about trailing white space.
+$FGREP ': all.test' stdout | sed "s/[$sp$tab]*$//" > got
+
+cat > exp <<END
+PASS: all.test 1 # TDO
+FAIL: all.test 2 # SKP
+PASS: all.test 3 #SIKP
+FAIL: all.test 4 # TOD
+PASS: all.test 5 # ${tab}${tab}TOOD
+END
+
+cat exp
+cat got
+diff exp got
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Auxiliary shell script that copies the common data and files used by
+# many tests on TAP support into the current directory. It should be
+# sourced by client test scripts, and assumes the auxiliary test
+# `tap-common-setup.test' has been run beforehand (it will error out
+# if this is not the case).
+
+# Check that we are running from a proper directory: last thing we want
+# is to overwrite some random user files.
+test -f ../defs-static && test -f ../defs \
+ || fatal_ "running from a wrong directory"
+
+test ! -f Makefile.am || mv Makefile.am Makefile.am~ \
+ || fatal_ "failed to save Makefile.am"
+
+test -d ../tap-common-setup.dir && cp -Rp ../tap-common-setup.dir/* . \
+ || fatal_ "couldn't get precomputed data files"
+
+fetch_tap_driver
+
+if test -f Makefile.am~; then
+ mv -f Makefile.am~ Makefile.am \
+ || fatal_ "failed to restore Makefile.am"
+ echo 'TEST_LOG_DRIVER = $(PERL) $(srcdir)/tap-driver' >> Makefile.am \
+ || fatal_ "failed to update Makefile.am"
+ $AUTOMAKE Makefile \
+ || fatal_ "failed to remake Makefile.in"
+ ./config.status Makefile \
+ || fatal_ "failed to remake Makefile"
+fi
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - a test script terminated by a signal causes an hard error
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+echo TESTS = > Makefile.am
+for sig in 1 2 13 15; do
+ unindent > signal-$sig.test <<END
+ #!/bin/sh
+ echo 1..1
+ echo ok 1
+ kill -$sig \$\$
+ echo "Bail out! \$0 not killed?"
+END
+ echo TESTS += signal-$sig.test >> Makefile.am
+done
+
+chmod a+x *.test
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+# Solaris /bin/sh, when killed with a SIGTERM or SIGINT signal, can end up
+# exiting with exit status 208, instead of leaving the correct wide exit
+# status to the parent. See:
+# <http://dbaspot.com/shell/396118-bourne-shell-exit-code-term.html>
+# We need to detect and work around this incompatibility.
+
+if /bin/sh -c 'kill -2 $$'; then
+ fatal_ "/bin/sh cannot kill itself"
+elif test $? -eq 208; then
+ have_solaris_bug=yes
+else
+ have_solaris_bug=no
+fi
+
+signal_caught ()
+{
+ numeric=$1
+ symbolic=$2
+ sig_re="((SIG)?$symbolic|$numeric)"
+ wbound_re="($|[^a-zA-Z0-9_-])"
+ pfx_re="^ERROR: signal-$numeric\\.test"
+ case $am_tap_implementation in
+ perl) rx="$pfx_re - terminated by signal $sig_re$";;
+ shell) rx="$pfx_re .*terminated by signal $sig_re$wbound_re";;
+ *) fatal_ "invalid \$am_tap_implementation '$am_tap_implementation'";;
+ esac
+ LC_ALL=C $EGREP "$rx" stdout && return 0
+ case $have_solaris_bug,$symbolic in
+ yes,INT|yes,TERM)
+ $EGREP "$pfx_re - exited with status 208( |$)" stdout && return 0;;
+ esac
+ return 1
+}
+
+all_signals_caught ()
+{
+ # These are the only signals that are portably trappable.
+ signal_caught 1 HUP
+ signal_caught 2 INT
+ signal_caught 13 PIPE
+ signal_caught 15 TERM
+}
+
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+count_test_results total=8 pass=4 fail=0 xpass=0 xfail=0 skip=0 error=4
+all_signals_caught
+
+echo 'TEST_LOG_DRIVER_FLAGS = --ignore-exit' >> Makefile
+$MAKE check >stdout || { cat stdout; Exit 1; }
+cat stdout
+count_test_results total=4 pass=4 fail=0 xpass=0 xfail=0 skip=0 error=0
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Auxiliary script for tests on TAP support: checking testsuite summary.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+br='============================================================================'
+
+case $use_colors in
+ yes|no) ;;
+ *) fatal_ "invalid \$use_colors value '$use_colors'"
+esac
+
+fetch_tap_driver
+
+cat > configure.in <<END
+AC_INIT([GNU AutoTAP], [5.12], [bug-automake@gnu.org])
+AM_INIT_AUTOMAKE([parallel-tests])
+AC_CONFIG_FILES([Makefile])
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+TEST_LOG_DRIVER = $(srcdir)/tap-driver
+TEST_LOG_COMPILER = cat
+TESTS = all.test
+END
+
+# The following shell variables are influential for this function:
+# - expect_failure
+# - use_colors
+do_check ()
+{
+ case $#,$1 in
+ 1,--pass) expect_failure=no;;
+ 1,--fail) expect_failure=yes;;
+ *) fatal_ "invalid usage of 'do_check'";;
+ esac
+ shift
+ cat > summary.exp
+ cat all.test
+ st=0
+ if test $use_colors = yes; then
+ make_cmd="env AM_COLOR_TESTS=always $MAKE -e"
+ else
+ make_cmd=$MAKE
+ fi
+ $make_cmd check > stdout || st=$?
+ cat stdout
+ if test $expect_failure = yes; then
+ test $st -gt 0 || Exit 1
+ else
+ test $st -eq 0 || Exit 1
+ fi
+ $PERL -w "$testsrcdir"/extract-testsuite-summary stdout > summary.got \
+ || fatal_ "cannot extract testsuite summary"
+ cat summary.exp
+ cat summary.got
+ if test $use_colors = yes; then
+ # Use cmp, not diff, because the files might contain binary data.
+ compare=cmp
+ else
+ compare=diff
+ fi
+ $compare summary.exp summary.got || Exit 1
+}
+
+if test $use_colors = yes; then
+ red='\e[0;31m'
+ grn='\e[0;32m'
+ lgn='\e[1;32m'
+ blu='\e[1;34m'
+ mgn='\e[0;35m'
+ brg='\e[1m'
+ std='\e[m'
+ echo AUTOMAKE_OPTIONS = color-tests >> Makefile.am
+else
+ red= grn= lgn= blu= mgn= brg= std=
+fi
+
+ success_header="\
+${grn}${br}${std}
+${grn}Testsuite summary for GNU AutoTAP 5.12${std}
+${grn}${br}${std}"
+
+ success_footer=${grn}${br}${std}
+
+ failure_header="\
+${red}${br}${std}
+${red}Testsuite summary for GNU AutoTAP 5.12${std}
+${red}${br}${std}"
+
+ failure_footer="\
+${red}${br}${std}
+${red}See ./test-suite.log${std}
+${red}Please report to bug-automake@gnu.org${std}
+${red}${br}${std}"
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE --add-missing
+
+./configure
+
+# 1 pass.
+{ echo 1..1 && echo ok; } > all.test
+do_check --pass <<END
+$success_header
+${brg}# TOTAL: 1${std}
+${grn}# PASS: 1${std}
+# SKIP: 0
+# XFAIL: 0
+# FAIL: 0
+# XPASS: 0
+# ERROR: 0
+$success_footer
+END
+
+# 1 skip.
+{ echo 1..1 && echo 'ok # SKIP'; } > all.test
+do_check --pass <<END
+$success_header
+${brg}# TOTAL: 1${std}
+# PASS: 0
+${blu}# SKIP: 1${std}
+# XFAIL: 0
+# FAIL: 0
+# XPASS: 0
+# ERROR: 0
+$success_footer
+END
+
+# 1 xfail.
+{ echo 1..1 && echo 'not ok # TODO'; } > all.test
+do_check --pass <<END
+$success_header
+${brg}# TOTAL: 1${std}
+# PASS: 0
+# SKIP: 0
+${lgn}# XFAIL: 1${std}
+# FAIL: 0
+# XPASS: 0
+# ERROR: 0
+$success_footer
+END
+
+# 1 fail.
+{ echo 1..1 && echo not ok; } > all.test
+do_check --fail <<END
+$failure_header
+${brg}# TOTAL: 1${std}
+# PASS: 0
+# SKIP: 0
+# XFAIL: 0
+${red}# FAIL: 1${std}
+# XPASS: 0
+# ERROR: 0
+$failure_footer
+END
+
+# 1 xpass.
+{ echo 1..1 && echo 'ok # TODO'; } > all.test
+do_check --fail <<END
+$failure_header
+${brg}# TOTAL: 1${std}
+# PASS: 0
+# SKIP: 0
+# XFAIL: 0
+# FAIL: 0
+${red}# XPASS: 1${std}
+# ERROR: 0
+$failure_footer
+END
+
+# 1 hard error.
+{ echo 1..1 && echo 'Bail out!'; } > all.test
+do_check --fail <<END
+$failure_header
+${brg}# TOTAL: 1${std}
+# PASS: 0
+# SKIP: 0
+# XFAIL: 0
+# FAIL: 0
+# XPASS: 0
+${mgn}# ERROR: 1${std}
+$failure_footer
+END
+
+# 3 non-failing results.
+cat > all.test <<END
+1..3
+ok
+not ok # TODO
+ok # SKIP
+END
+do_check --pass <<END
+$success_header
+${brg}# TOTAL: 3${std}
+${grn}# PASS: 1${std}
+${blu}# SKIP: 1${std}
+${lgn}# XFAIL: 1${std}
+# FAIL: 0
+# XPASS: 0
+# ERROR: 0
+$success_footer
+END
+
+# 1 pass, 1 skip, 1 fail.
+cat > all.test <<END
+1..3
+ok
+ok # SKIP
+not ok
+END
+do_check --fail <<END
+$failure_header
+${brg}# TOTAL: 3${std}
+${grn}# PASS: 1${std}
+${blu}# SKIP: 1${std}
+# XFAIL: 0
+${red}# FAIL: 1${std}
+# XPASS: 0
+# ERROR: 0
+$failure_footer
+END
+
+# 1 pass, 1 xfail, 1 xpass.
+cat > all.test <<END
+1..3
+ok
+ok # TODO
+not ok # TODO
+END
+do_check --fail <<END
+$failure_header
+${brg}# TOTAL: 3${std}
+${grn}# PASS: 1${std}
+# SKIP: 0
+${lgn}# XFAIL: 1${std}
+# FAIL: 0
+${red}# XPASS: 1${std}
+# ERROR: 0
+$failure_footer
+END
+
+# 1 skip, 1 xfail, 1 error.
+cat > all.test <<END
+1..3
+ok # SKIP
+not ok # TODO
+Bail out!
+END
+do_check --fail <<END
+$failure_header
+${brg}# TOTAL: 3${std}
+# PASS: 0
+${blu}# SKIP: 1${std}
+${lgn}# XFAIL: 1${std}
+# FAIL: 0
+# XPASS: 0
+${mgn}# ERROR: 1${std}
+$failure_footer
+END
+
+# 1 of each kind
+cat > all.test <<END
+1..6
+ok
+not ok
+ok # TODO
+not ok # TODO
+ok # SKIP
+Bail out!
+END
+do_check --fail <<END
+$failure_header
+${brg}# TOTAL: 6${std}
+${grn}# PASS: 1${std}
+${blu}# SKIP: 1${std}
+${lgn}# XFAIL: 1${std}
+${red}# FAIL: 1${std}
+${red}# XPASS: 1${std}
+${mgn}# ERROR: 1${std}
+$failure_footer
+END
+
+# Prepare some common data for later.
+for i in 0 1 2 3 4 5 6 7 8 9; do
+ for j in 0 1 2 3 4 5 6 7 8 9; do
+ echo "ok"
+ echo "not ok # TODO"
+ echo "ok # SKIP"
+ done
+done > tap
+
+# Lots of non-failures (300 per kind).
+(cat tap && cat tap && cat tap) > all.test
+test `wc -l <all.test` -eq 900 || Exit 99 # Sanity check.
+echo 1..900 >> all.test # Test plan.
+do_check --pass <<END
+$success_header
+${brg}# TOTAL: 900${std}
+${grn}# PASS: 300${std}
+${blu}# SKIP: 300${std}
+${lgn}# XFAIL: 300${std}
+# FAIL: 0
+# XPASS: 0
+# ERROR: 0
+$success_footer
+END
+
+# 1 failure and lots of non-failures means failure.
+(cat tap && echo "not ok" && cat tap) > all.test
+test `wc -l <all.test` -eq 601 || Exit 99 # Sanity check.
+echo 1..601 >> all.test # Test plan.
+do_check --fail <<END
+$failure_header
+${brg}# TOTAL: 601${std}
+${grn}# PASS: 200${std}
+${blu}# SKIP: 200${std}
+${lgn}# XFAIL: 200${std}
+${red}# FAIL: 1${std}
+# XPASS: 0
+# ERROR: 0
+$failure_footer
+END
+
+# 1 error and lots of non-failures means failure.
+(cat tap && sed 30q tap && echo 'Bail out!') > all.test
+test `wc -l <all.test` -eq 331 || Exit 99 # Sanity check.
+echo 1..331 >> all.test # Test plan.
+do_check --fail <<END
+$failure_header
+${brg}# TOTAL: 331${std}
+${grn}# PASS: 110${std}
+${blu}# SKIP: 110${std}
+${lgn}# XFAIL: 110${std}
+# FAIL: 0
+# XPASS: 0
+${mgn}# ERROR: 1${std}
+$failure_footer
+END
+
+:
#! /bin/sh
-# Copyright (C) 1996, 2000, 2001, 2002, 2010, 2011 Free Software
-# Foundation, Inc.
+# Copyright (C) 2011 Free Software Foundation, Inc.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Test for bug reported by François Pinard.
+# TAP support:
+# - colorized testsuite summary
-. ./defs || Exit 1
+. ./defs-static
-cat > configure.in << 'END'
-AC_INIT
-AM_INIT_AUTOMAKE(nonesuch, nonesuch)
-AC_OUTPUT([Makefile],
-[echo zardoz has spoken])
-END
-
-: > Makefile.am
-
-$ACLOCAL
-$AUTOMAKE
+use_colors=yes
+. "$testsrcdir"/tap-summary-aux.sh
#! /bin/sh
-# Copyright (C) 1998, 2001, 2002, 2010, 2011 Free Software Foundation,
-# Inc.
+# Copyright (C) 2011 Free Software Foundation, Inc.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Make sure conditionals work with man pages.
+# TAP support:
+# - colorized testsuite summary
-. ./defs || Exit 1
+. ./defs-static
-cat >> configure.in << 'END'
-AM_CONDITIONAL([FRED], [true])
-END
-
-cat > Makefile.am << 'END'
-if FRED
-man_MANS = foo.1
-else
-man_MANS = joe.1
-endif
-END
-
-$ACLOCAL
-$AUTOMAKE
-
-:
+use_colors=no
+. "$testsrcdir"/tap-summary-aux.sh
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - a test result numbered as 0 is to be considered out-of-order
+# This is consistent with the behaviour of the `prove' utility.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > a.test <<END
+1..1
+ok 0
+END
+
+cat > b.test <<END
+1..1
+not ok 0
+END
+
+cat > c.test <<END
+1..1
+ok 0 foo # SKIP
+END
+
+cat > d.test <<END
+1..1
+not ok 0 bar # TODO
+END
+
+cat > e.test <<END
+1..1
+ok 0 # TODO
+END
+
+TESTS='a.test b.test c.test d.test e.test' $MAKE -e check >stdout \
+ && { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=5 pass=0 fail=0 xpass=0 xfail=0 skip=0 error=5
+
+grep '^ERROR: a\.test 0 # OUT-OF-ORDER (expecting 1)$' stdout
+grep '^ERROR: b\.test 0 # OUT-OF-ORDER (expecting 1)$' stdout
+grep '^ERROR: c\.test 0 foo # OUT-OF-ORDER (expecting 1)$' stdout
+grep '^ERROR: d\.test 0 bar # OUT-OF-ORDER (expecting 1)$' stdout
+grep '^ERROR: e\.test 0 # OUT-OF-ORDER (expecting 1)$' stdout
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - TODO and SKIP directives on the same line: the first one wins
+# See also related test 'tap-ambiguous-directive.test'.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > all.test <<END
+1..2
+ok 1 # SKIP TODO
+not ok 2 # TODO SKIP
+END
+
+$MAKE check >stdout || { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=2 pass=0 fail=0 xpass=0 xfail=1 skip=1 error=0
+
+grep '^SKIP: all\.test 1 ' stdout
+grep '^XFAIL: all\.test 2 ' stdout
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - normalization of whitespace in console testsuite progress associated
+# with TODO and SKIP directives
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > stub.tap <<END
+1 # TODO
+2 # TODO foo?
+3 # TODO: bar!
+4 aa # TODO
+5 bb # TODO fnord 5
+6 cc # TODO:${tab}fnord 6
+7 - x # TODO
+8 - y # TODO fnord $tab 8
+9 - z # TODO: fnord 9 $tab
+10# TODO x0
+11$tab# TODO x1
+12 $tab$tab # TODO x2
+13 asd# TODO x3
+14 sad$tab# TODO x4
+15 das$tab$tab # TODO x5
+END
+
+cat > stub.exp <<END
+1 # TODO
+2 # TODO foo?
+3 # TODO: bar!
+4 aa # TODO
+5 bb # TODO fnord 5
+6 cc # TODO:${tab}fnord 6
+7 - x # TODO
+8 - y # TODO fnord $tab 8
+9 - z # TODO: fnord 9
+10 # TODO x0
+11 # TODO x1
+12 # TODO x2
+13 asd # TODO x3
+14 sad # TODO x4
+15 das # TODO x5
+END
+
+plan=1..15
+
+my_make_check ()
+{
+ xpass=0 xfail=0 skip=0
+ case $1 in
+ xpass|xfail|skip) eval $1=15;;
+ *) fatal_ "bad argument '$1' for my_make_check";;
+ esac
+ cat all.test
+ $MAKE check >stdout || : # Don't care about the exit status in this test.
+ cat stdout
+ count_test_results total=15 pass=0 fail=0 error=0 \
+ xpass=$xpass xfail=$xfail skip=$skip
+ # Don't be too strict w.r.t. possible normalization of "TODO: foo" into
+ # "TODO : foo" (as is done by, e.g., the 'TAP::Parser' perl module).
+ LC_ALL=C grep '^[A-Z][A-Z]*:' stdout \
+ | sed -e 's/# TODO *:/# TODO:/' -e 's/# SKIP *:/# SKIP:/' > got
+ cat exp
+ cat got
+ diff exp got
+}
+
+# For "TODO" directives leading to XPASS results.
+(echo $plan && sed -e 's/^/ok /' stub.tap) > all.test
+sed -e 's/^/XPASS: all.test /' stub.exp > exp
+my_make_check xpass
+
+# For "TODO" directives leading to XFAIL results.
+(echo $plan && sed -e 's/^/not ok /' stub.tap) > all.test
+sed -e 's/^/XFAIL: all.test /' stub.exp > exp
+my_make_check xfail
+
+# For "SKIP" directives.
+(echo $plan && sed -e 's/^/ok /' -e 's/TODO/SKIP/' stub.tap) > all.test
+sed -e 's/TODO/SKIP/' -e 's/^/SKIP: all.test /' stub.exp > exp
+my_make_check skip
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - TODO and SKIP directives are case-insensitive
+# - TODO and SKIP directives can be followed optionally by a colon ":"
+# and by an optional explanation.
+# - our driver isn't fooled into recognizing TODO and SKIP directives
+# spuriously
+# - the reasons for TODO and SKIP, if present, are nicely printed in
+# the testsuite progress output
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+# ----------------------------------------------------- #
+# Check all possible combinations of: #
+# - uppercase/lowercase #
+# - with/without colon character ":" #
+# - with/without explanatory message #
+# in TODO and SKIP directives. #
+# ----------------------------------------------------- #
+
+# There are 2 * 2^6 + 2 * 2^6 = 256 tests.
+echo 1..256 > all.test
+
+# These nested loops are clearer without indentation.
+
+for c1 in t T; do
+for c2 in o O; do
+for c3 in d D; do
+for c4 in o O; do
+for ex in '' ':' ' foo' ': foo'; do
+ echo "not ok # $c1$c2$c3$c4$ex"
+ echo "not ok# $c1$c2$c3$c4$ex"
+done; done; done; done; done >> all.test
+
+for c1 in s S; do
+for c2 in k K; do
+for c3 in i I; do
+for c4 in p P; do
+for ex in '' ':' ' foo' ': foo'; do
+ echo "ok # $c1$c2$c3$c4$ex"
+ echo "ok# $c1$c2$c3$c4$ex"
+done; done; done; done; done >> all.test
+
+cat all.test # For debugging.
+
+$MAKE check >stdout || { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=256 pass=0 fail=0 xpass=0 xfail=128 skip=128 error=0
+
+# -------------------------------------------------------- #
+# TODO ans SKIP directives aren't recognized spuriously. #
+# -------------------------------------------------------- #
+
+cat > all.test <<'END'
+1..9
+ok TODO
+ok - TODO
+ok 3 TODO
+ok 4 - TODO
+ok SKIP
+ok - SKIP
+ok 7 SKIP
+ok 8 - SKIP
+ok 9
+END
+
+$MAKE check >stdout || { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=9 pass=9 fail=0 xpass=0 xfail=0 skip=0 error=0
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - unplanned tests are properly reported as errors
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > all.test <<END
+1..1
+ok 1
+ok 2
+END
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+count_test_results total=3 pass=1 fail=0 xpass=0 xfail=0 skip=0 error=2
+grep '^ERROR: all\.test - too many tests run (expected 1, got 2)$' stdout
+grep '^ERROR: all\.test 2 # UNPLANNED$' stdout
+
+cat > all.test <<END
+1..2
+ok 1
+ok 2
+ok 3
+END
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+count_test_results total=4 pass=2 fail=0 xpass=0 xfail=0 skip=0 error=2
+grep '^ERROR: all\.test - too many tests run (expected 2, got 3)$' stdout
+grep '^ERROR: all\.test 3 # UNPLANNED$' stdout
+
+# Interaction with XFAIL_TESTS.
+cat > all.test <<END
+1..2
+not ok 1
+ok 2 # SKIP
+ok 3
+not ok 4
+ok 5 # SKIP
+END
+XFAIL_TESTS=all.test $MAKE -e check >stdout && { cat stdout; Exit 1; }
+cat stdout
+count_test_results total=6 pass=0 fail=0 xpass=0 xfail=1 skip=1 error=4
+grep '^ERROR: all\.test - too many tests run (expected 2, got 5)$' stdout
+grep '^ERROR: all\.test 3 # UNPLANNED$' stdout
+grep '^ERROR: all\.test 4 # UNPLANNED$' stdout
+grep '^ERROR: all\.test 5 # UNPLANNED$' stdout
+
+cat > all.test <<END
+1..1
+ok 1
+
+ok
+ok 3
+ok foo
+ok 5 - bar bar
+
+not ok
+not ok 7
+not ok foo
+not ok 9 - bar bar
+
+ok # TODO
+ok 11 # TODO
+ok foo # TODO
+ok 13 - bar bar # TODO
+
+not ok # TODO
+not ok 15 # TODO
+not ok foo # TODO
+not ok 17 - bar bar # TODO
+
+ok # SKIP
+ok 19 # SKIP
+ok foo # SKIP
+ok 21 - bar bar # SKIP
+END
+
+cat > t <<END
+
+2
+3
+4 foo
+5 - bar bar
+
+6
+7
+8 foo
+9 - bar bar
+
+10
+11
+12 foo
+13 - bar bar
+
+14
+15
+16 foo
+17 - bar bar
+
+18
+19
+20 foo
+21 - bar bar
+
+END
+
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+count_test_results total=22 pass=1 fail=0 xpass=0 xfail=0 skip=0 error=21
+
+echo 'PASS: all.test 1' > exp
+sed -e '/^$/d' -e 's/.*/ERROR: all.test & # UNPLANNED/' t >> exp
+echo 'ERROR: all.test - too many tests run (expected 1, got 21)' >> exp
+
+$FGREP ': all.test' stdout > got
+
+cat exp
+cat got
+diff exp got
+
+# Note that, if the TAP input has a trailing plan, it is not possible
+# to flag unplanned tests as such, since we do not know they're unplanned
+# until the plan is reached; still, we should give at least an error
+# message about the unmatched number of tests once we've got the plan.
+
+for x in 'ok' 'ok 3' 'not ok' 'not ok # TODO' 'ok # TODO' 'ok # SKIP'; do
+ unindent > all.test <<END
+ ok 1
+ ok 2 # SKIP
+ $x
+ 1..2
+END
+ $MAKE check >stdout && { cat stdout; Exit 1; }
+ cat stdout
+ test `$FGREP -c ': all.test' stdout` -eq 4
+ $EGREP '^PASS: all\.test 1($| )' stdout
+ $EGREP '^SKIP: all\.test 2($| )' stdout
+ $EGREP ': all\.test 3($| )' stdout
+ grep '^ERROR: all\.test - too many tests run (expected 2, got 3)$' stdout
+done
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support: whitespace normalization (or lack thereof) in the testsuite
+# progress output on console. We keep all these checks here in a single
+# script so that a potential cosmetic change in the output format won't
+# force us to tweak dozens of other tests (hopefully).
+# See also related test 'tap-todo-skip-whitespace.test'.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat > Makefile.am << 'END'
+TEST_LOG_COMPILER = cat
+TESTS =
+END
+
+: > exp
+
+spaces_a=${sp}${tab}${tab}${sp}${sp}${tab}
+spaces_b=${tab}${tab}${sp}${tab}${sp}${sp}${sp}
+
+#-----------------------------------------------------------------------
+
+echo TESTS += numbers.test >> Makefile.am
+
+cat > numbers.test <<END
+1..6
+ok${spaces_a}1
+ok 2${spaces_b}
+ok${spaces_a}3${spaces_b}
+not ok${spaces_b}4
+not ok 5${spaces_a}
+not ok${spaces_b}6${spaces_a}
+END
+
+cat >> exp <<END
+PASS: numbers.test 1
+PASS: numbers.test 2
+PASS: numbers.test 3
+FAIL: numbers.test 4
+FAIL: numbers.test 5
+FAIL: numbers.test 6
+END
+
+#-----------------------------------------------------------------------
+
+echo TESTS += description.test >> Makefile.am
+
+cat > description.test <<END
+1..8
+ok${spaces_a}+foo
+ok +bar${spaces_b}
+ok${spaces_a}+baz${spaces_b}
+not ok${spaces_b}-foo
+not ok -bar${spaces_a}
+not ok${spaces_b}-baz${spaces_a}
+ok u${spaces_b}v${spaces_a}w${sp}
+not ok${spaces_a}x${spaces_a}y${tab}z${tab}
+END
+
+cat >> exp <<END
+PASS: description.test 1 +foo
+PASS: description.test 2 +bar
+PASS: description.test 3 +baz
+FAIL: description.test 4 -foo
+FAIL: description.test 5 -bar
+FAIL: description.test 6 -baz
+PASS: description.test 7 u${spaces_b}v${spaces_a}w
+FAIL: description.test 8 x${spaces_a}y${tab}z
+END
+
+#-----------------------------------------------------------------------
+
+# "Bail out!" magic.
+
+echo TESTS += bailout.test >> Makefile.am
+
+cat > bailout.test <<END
+1..1
+Bail out!${tab}${sp}${sp}${tab}We're out of disk space.
+ok 1
+END
+
+cat >> exp <<END
+ERROR: bailout.test - Bail out! We're out of disk space.
+END
+
+echo TESTS += bailout2.test >> Makefile.am
+
+cat > bailout2.test <<END
+1..1
+Bail out!foo${tab}${sp}
+ok 1
+END
+
+cat >> exp <<END
+ERROR: bailout2.test - Bail out! foo
+END
+
+#-----------------------------------------------------------------------
+
+# Diagnostic lines.
+
+echo AM_TEST_LOG_DRIVER_FLAGS = --comments >> Makefile.am
+echo TESTS += cmnt.test >> Makefile.am
+
+cat > cmnt.test <<END
+1..1
+ok 1
+#Leading whitespace gets added
+# ${tab}${tab} ${tab}Extra leading whitespace is stripped
+# Trailing whitespace is stripped ${tab} ${tab}${tab}
+# Middle${tab}whitespace is${tab} ${tab}${tab} kept
+# ${tab} And only${tab}middle ${tab}whitespace ${tab}${tab} ${tab}
+END
+
+cat >> exp <<END
+PASS: cmnt.test 1
+# cmnt.test: Leading whitespace gets added
+# cmnt.test: Extra leading whitespace is stripped
+# cmnt.test: Trailing whitespace is stripped
+# cmnt.test: Middle${tab}whitespace is${tab} ${tab}${tab} kept
+# cmnt.test: And only${tab}middle ${tab}whitespace
+END
+
+#-----------------------------------------------------------------------
+
+# TODO: we should have more checks here ... (but let's not over-do FTM).
+
+#-----------------------------------------------------------------------
+
+chmod a+x *.test
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+# Don't care about exit status or number of test results, they should be
+# checked for in many other tests.
+$MAKE check >stdout || :
+cat stdout
+
+LC_ALL=C sort exp > t
+mv -f t exp
+
+# We need the sort below to account for parallel make usage.
+LC_ALL=C grep '[a-z0-9][a-z0-9]*\.test' stdout | LC_ALL=C sort > got
+
+cat exp
+cat got
+diff exp got
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - numbered and unnumbered test results can coexist
+# - tests without explicit number get automatically numbered in the
+# testsuite progress output on console
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > all.test <<'END'
+1..7
+ok 1 foo
+ok 2 - foo2
+ok - bar
+not ok baz # TODO
+not ok 5 - asd # TODO
+ok 6 # SKIP
+ok zardoz
+END
+
+$MAKE check >stdout || { cat stdout; Exit 1; }
+cat stdout
+count_test_results total=7 pass=4 fail=0 xpass=0 xfail=2 skip=1 error=0
+
+grep '^PASS: all\.test 1 foo$' stdout
+grep '^PASS: all\.test 2 - foo2$' stdout
+grep '^PASS: all\.test 3 - bar$' stdout
+grep '^XFAIL: all\.test 4 baz # TODO$' stdout
+grep '^XFAIL: all\.test 5 - asd # TODO$' stdout
+grep '^SKIP: all\.test 6 # SKIP$' stdout
+grep '^PASS: all\.test 7 zardoz$' stdout
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP and $(XFAIL_TESTS): test results without directives are turned from
+# PASS to XPASS and from FAIL to XFAIL; other results are unchanged.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+echo 'XFAIL_TESTS = $(TESTS)' >> Makefile
+
+cat > all.test <<END
+1..6
+ok 1
+not ok 2
+ok 3 # TODO
+not ok 4 # TODO
+ok 5 # SKIP
+# The next should count as a failure (and thus will be
+# rendered into an expected failure by XFAIL_TESTS).
+not ok 6 # SKIP
+Bail out!
+END
+
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=7 pass=0 fail=0 xpass=2 xfail=3 skip=1 error=1
+
+grep '^XPASS: all\.test 1$' stdout
+grep '^XFAIL: all\.test 2$' stdout
+grep '^XPASS: all\.test 3 # TODO' stdout
+grep '^XFAIL: all\.test 4 # TODO' stdout
+grep '^SKIP: all\.test 5 # SKIP' stdout
+grep '^XFAIL: all\.test 6 # SKIP' stdout
+grep '^ERROR: all\.test - Bail out!' stdout
+
+# Check that the exit status of the testsuite is indeed 0 when we
+# would expect success.
+
+cat > all.test <<END
+1..3
+not ok 1
+ok 2 # SKIP
+not ok 3 # TODO
+END
+
+$MAKE check >stdout || { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=3 pass=0 fail=0 xpass=0 xfail=2 skip=1 error=0
+
+:
$AUTOMAKE
./configure
-grep 'am__tar.*false' Makefile && Exit 77
+if grep 'am__tar.*false' Makefile; then
+ skip_ "cannot find proper archiver program"
+fi
$MAKE distcheck
test -f tar-1.0.tar.gz
$AUTOMAKE
./configure
-grep 'am__tar.*false' Makefile && Exit 77
+if grep 'am__tar.*false' Makefile; then
+ skip_ "cannot find proper archiver program"
+fi
$MAKE distcheck
test -f tar2-1.0.tar.gz
# Test to make sure target specific CFLAGS work
# Assar Westerlund <assar@sics.se>
-required=gcc
+required=cc
. ./defs || Exit 1
cat >> configure.in << 'END'
cat > foo.c << 'END'
#include <stdio.h>
#ifdef FOO
-int main(void) {
+int main(void)
+{
return 0;
}
#endif
cd build
../configure
$MAKE
-./foo
-./bar
+if cross_compiling; then :; else
+ ./foo
+ ./bar
+fi
cd ..
./configure
$MAKE
-./foo
-./bar
+if cross_compiling; then :; else
+ ./foo
+ ./bar
+fi
:
# Check that target clashes are diagnosed.
-required=gcc
. ./defs || Exit 1
cat >> configure.in << 'END'
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Custom test drivers: if a log file has a directory component (as in
+# e.g., `sub/foo.log'), the Automake test harness must ensure that
+# directory exists before calling any custom test driver.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+# The extra `.' components below are meant.
+TESTS = sub/foo sub/subsub/bar.test ././sub1/./baz
+$(TESTS):
+LOG_DRIVER = $(srcdir)/checkdir-driver
+TEST_LOG_DRIVER = $(LOG_DRIVER)
+EXTRA_DIST = checkdir-driver
+
+check-local: $(TEST_SUITE_LOG)
+ test -d sub
+ test -d sub1
+ test -d sub/subsub
+ test -f sub/foo.log
+ test -f sub/subsub/bar.log
+ test -f sub1/baz.log
+ test -f sub/foo.trs
+ test -f sub/subsub/bar.trs
+ test -f sub1/baz.trs
+END
+
+cat > checkdir-driver <<'END'
+#! /bin/sh
+set -e; set -u
+while test $# -gt 0; do
+ case $1 in
+ --log-file) log_file=$2; shift;;
+ --trs-file) trs_file=$2; shift;;
+ --test-name|--expect-failure|--color-tests|--enable-hard-errors) shift;;
+ --) shift; break;;
+ *) echo "$0: invalid option/argument: '$1'" >&2; exit 2;;
+ esac
+ shift
+done
+echo "log: $log_file" # For debugging.
+echo "trs: $trs_file" # Ditto.
+case $log_file in */*);; *) exit 1;; esac
+dir_log=`expr "$log_file" : '\(.*\)/[^/]*'`
+dir_trs=`expr "$trs_file" : '\(.*\)/[^/]*'`
+echo "dir_log: $dir_log" # For debugging.
+echo "dir_trs: $dir_trs" # Likewise.
+test x"$dir_trs" = x"$dir_log" || exit 1
+test -d "$dir_log" || exit 1
+echo dummy1 > "$log_file"
+echo dummy2 > "$trs_file"
+END
+chmod a+x checkdir-driver
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE
+
+./configure
+$MAKE check
+$MAKE distcheck
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Custom test drivers features:
+# - check-html
+# - recheck-html
+# Keep this in sync with sister test `parallel-tests2.test'.
+
+parallel_tests=yes
+required=rst2html
+. ./defs || Exit 1
+
+cp "$testsrcdir"/trivial-test-driver . \
+ || fatal_ "failed to fetch auxiliary script trivial-test-driver"
+
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+TEST_LOG_DRIVER = $(SHELL) ./trivial-test-driver
+TEST_SUITE_LOG = mylog.log
+TESTS = foo.test bar.test baz.test
+check_SCRIPTS = bla
+bla:
+ echo bla > $@
+CLEANFILES = bla
+END
+
+cat > foo.test <<'END'
+#! /bin/sh
+if test -f bla; the
+ echo "PASS: this is $0"
+else
+ echo "FAIL: this is $0"
+fi
+END
+
+cat > bar.test <<'END'
+#! /bin/sh
+echo "FAIL: this is $0"
+END
+
+cat > baz.test <<'END'
+#! /bin/sh
+echo "FAIL: this is $0"
+END
+
+chmod a+x foo.test bar.test baz.test
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE
+
+./configure
+
+$MAKE check-html && Exit 1
+test -f mylog.html
+# check-html should cause check_SCRIPTS to be created.
+test -f bla
+
+# "make clean" should remove HTML files.
+$MAKE clean
+test ! -f mylog.html
+test ! -f bla
+
+# Always create the HTML output, even if there were no failures.
+rm -f mylog.html
+env TESTS=foo.test $MAKE -e check-html
+test -f mylog.html
+
+# Create summarizing HTML output also with recheck-html.
+rm -f mylog.html
+env TESTS=foo.test $MAKE -e recheck-html
+test -f mylog.html
+
+# check-html should cause check_SCRIPTS to be created.
+$MAKE clean
+env TEST_LOGS=foo.log $MAKE -e check-html
+test -f bla
+test -f foo.log
+test -f mylog.html
+# recheck-html should cause check_SCRIPTS to be created, and should rerun
+# no tests if it appears that check has not been run.
+$MAKE clean
+env TESTS=foo.test $MAKE -e recheck-html
+test -f bla
+test ! -f foo.log
+test -f mylog.html
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Custom test drivers: try the "recheck" functionality with test protocols
+# that allow multiple testcases in a single test script. This test not
+# only checks implementation details in Automake's custom test drivers
+# support, but also serves as a "usability test" for our APIs.
+# See also related tests `test-driver-custom-multitest-recheck2.test'
+# and `parallel-tests-recheck-override.test'.
+# Keep in sync with 'tap-recheck.test'.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cp "$testsrcdir"/trivial-test-driver . \
+ || fatal_ "failed to fetch auxiliary script trivial-test-driver"
+
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+TEST_LOG_DRIVER = $(SHELL) $(srcdir)/trivial-test-driver
+TESTS = a.test b.test c.test d.test
+END
+
+cat > a.test << 'END'
+#! /bin/sh
+echo PASS: aa
+echo PASS: AA
+: > a.run
+END
+
+cat > b.test << 'END'
+#! /bin/sh
+echo PASS:
+if test -f b.ok; then
+ echo PASS:
+else
+ echo ERROR:
+fi
+: > b.run
+END
+
+cat > c.test << 'END'
+#! /bin/sh
+if test -f c.pass; then
+ echo PASS: c0
+else
+ echo FAIL: c0
+fi
+if test -f c.xfail; then
+ echo XFAIL: c1
+else
+ echo XPASS: c1
+fi
+echo XFAIL: c2
+: > c.run
+END
+
+cat > d.test << 'END'
+#! /bin/sh
+echo SKIP: who cares ...
+(. ./d.extra) || echo FAIL: d.extra failed
+: > d.run
+END
+
+chmod a+x *.test
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE
+
+do_recheck ()
+{
+ case $* in
+ --fail) on_bad_rc='&&';;
+ --pass) on_bad_rc='||';;
+ *) fatal_ "invalid usage of function 'do_recheck'";;
+ esac
+ rm -f *.run
+ eval "\$MAKE recheck >stdout $on_bad_rc { cat stdout; ls -l; Exit 1; }; :"
+ cat stdout; ls -l
+}
+
+for vpath in : false; do
+ if $vpath; then
+ mkdir build
+ cd build
+ srcdir=..
+ else
+ srcdir=.
+ fi
+
+ $srcdir/configure
+
+ : A "make recheck" in a clean tree should run no tests.
+ do_recheck --pass
+ cat test-suite.log
+ test ! -r a.run
+ test ! -r a.log
+ test ! -r b.run
+ test ! -r b.log
+ test ! -r c.run
+ test ! -r c.log
+ test ! -r d.run
+ test ! -r d.log
+ count_test_results total=0 pass=0 fail=0 xpass=0 xfail=0 skip=0 error=0
+
+ : Run the tests for the first time.
+ $MAKE check >stdout && { cat stdout; Exit 1; }
+ cat stdout
+ ls -l
+ # All the test scripts should have run.
+ test -f a.run
+ test -f b.run
+ test -f c.run
+ test -f d.run
+ count_test_results total=9 pass=3 fail=2 xpass=1 xfail=1 skip=1 error=1
+
+ : Let us make b.test pass.
+ echo OK > b.ok
+ do_recheck --fail
+ # a.test has been successful the first time, so no need to re-run it.
+ # Similar considerations apply to similar checks, below.
+ test ! -r a.run
+ test -f b.run
+ test -f c.run
+ test -f d.run
+ count_test_results total=7 pass=2 fail=2 xpass=1 xfail=1 skip=1 error=0
+
+ : Let us make the first part of c.test pass.
+ echo OK > c.pass
+ do_recheck --fail
+ test ! -r a.run
+ test ! -r b.run
+ test -f c.run
+ test -f d.run
+ count_test_results total=5 pass=1 fail=1 xpass=1 xfail=1 skip=1 error=0
+
+ : Let us make also the second part of c.test pass.
+ echo KO > c.xfail
+ do_recheck --fail
+ test ! -r a.run
+ test ! -r b.run
+ test -f c.run
+ test -f d.run
+ count_test_results total=5 pass=1 fail=1 xpass=0 xfail=2 skip=1 error=0
+
+ : Nothing changed, so only d.test should be run.
+ for i in 1 2; do
+ do_recheck --fail
+ test ! -r a.run
+ test ! -r b.run
+ test ! -r c.run
+ test -f d.run
+ count_test_results total=2 pass=0 fail=1 xpass=0 xfail=0 skip=1 error=0
+ done
+
+ : Let us make d.test run more testcases, and experience _more_ failures.
+ unindent > d.extra <<'END'
+ echo SKIP: s
+ echo FAIL: f 1
+ echo PASS: p 1
+ echo FAIL: f 2
+ echo XPASS: xp
+ echo FAIL: f 3
+ echo FAIL: f 4
+ echo ERROR: e 1
+ echo PASS: p 2
+ echo ERROR: e 2
+END
+ do_recheck --fail
+ test ! -r a.run
+ test ! -r b.run
+ test ! -r c.run
+ test -f d.run
+ count_test_results total=11 pass=2 fail=4 xpass=1 xfail=0 skip=2 error=2
+
+ : Let us finally make d.test pass.
+ echo : > d.extra
+ do_recheck --pass
+ test ! -r a.run
+ test ! -r b.run
+ test ! -r c.run
+ test -f d.run
+ count_test_results total=1 pass=0 fail=0 xpass=0 xfail=0 skip=1 error=0
+
+ : All tests have been successful or skipped, nothing should be re-run.
+ do_recheck --pass
+ test ! -r a.run
+ test ! -r b.run
+ test ! -r c.run
+ test ! -r d.run
+ count_test_results total=0 pass=0 fail=0 xpass=0 xfail=0 skip=0 error=0
+
+ cd $srcdir
+
+done
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Custom test drivers: try the "recheck" functionality with test protocols
+# that allow multiple testcases in a single test script. In particular,
+# check that this still works when we override $(TESTS) and $(TEST_LOGS)
+# at make runtime.
+# See also related tests `test-driver-custom-multitest-recheck.test' and
+# `parallel-tests-recheck-override.test'.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cp "$testsrcdir"/trivial-test-driver . \
+ || fatal_ "failed to fetch auxiliary script trivial-test-driver"
+
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+TEST_LOG_DRIVER = $(SHELL) $(srcdir)/trivial-test-driver
+TESTS = a.test b.test c.test
+END
+
+cat > a.test << 'END'
+#! /bin/sh
+echo PASS: 1
+echo PASS: 2
+: > a.run
+END
+
+cat > b.test << 'END'
+#! /bin/sh
+echo SKIP: b0
+if test -f b.ok; then
+ echo XFAIL: b1
+else
+ echo FAIL: b2
+fi
+: > b.run
+END
+
+cat > c.test << 'END'
+#! /bin/sh
+if test -f c.err; then
+ echo ERROR: xxx
+elif test -f c.ok; then
+ echo PASS: ok
+else
+ echo XPASS: xp
+fi
+: > c.run
+END
+
+chmod a+x *.test
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE
+
+for vpath in : false; do
+ if $vpath; then
+ mkdir build
+ cd build
+ srcdir=..
+ else
+ srcdir=.
+ fi
+
+ $srcdir/configure
+
+ : Run the tests for the first time.
+ $MAKE check >stdout && { cat stdout; Exit 1; }
+ cat stdout
+ # All the test scripts should have run.
+ test -f a.run
+ test -f b.run
+ test -f c.run
+ count_test_results total=5 pass=2 fail=1 xpass=1 xfail=0 skip=1 error=0
+
+ rm -f *.run
+
+ : An empty '$(TESTS)' or '$(TEST_LOGS)' means that no test should be run.
+ for var in TESTS TEST_LOGS; do
+ env "$var=" $MAKE -e recheck >stdout || { cat stdout; Exit 1; }
+ cat stdout
+ count_test_results total=0 pass=0 fail=0 xpass=0 xfail=0 skip=0 error=0
+ test ! -r a.run
+ test ! -r b.run
+ test ! -r c.run
+ done
+ unset var
+
+ : a.test was sucessfull the first time, no need to re-run it.
+ env TESTS=a.test $MAKE -e recheck >stdout \
+ || { cat stdout; Exit 1; }
+ cat stdout
+ count_test_results total=0 pass=0 fail=0 xpass=0 xfail=0 skip=0 error=0
+ test ! -r a.run
+ test ! -r b.run
+ test ! -r c.run
+
+ : b.test failed, it should be re-run. And make it pass this time.
+ echo OK > b.ok
+ TEST_LOGS=b.log $MAKE -e recheck >stdout \
+ || { cat stdout; Exit 1; }
+ cat stdout
+ test ! -r a.run
+ test -f b.run
+ test ! -r c.run
+ count_test_results total=2 pass=0 fail=0 xpass=0 xfail=1 skip=1 error=0
+
+ rm -f *.run
+
+ : No need to re-run a.test or b.test anymore.
+ TEST_LOGS=b.log $MAKE -e recheck >stdout \
+ || { cat stdout; Exit 1; }
+ cat stdout
+ count_test_results total=0 pass=0 fail=0 xpass=0 xfail=0 skip=0 error=0
+ test ! -r a.run
+ test ! -r b.run
+ test ! -r c.run
+ TESTS='a.test b.test' $MAKE -e recheck >stdout \
+ || { cat stdout; Exit 1; }
+ cat stdout
+ count_test_results total=0 pass=0 fail=0 xpass=0 xfail=0 skip=0 error=0
+ test ! -r a.run
+ test ! -r b.run
+ test ! -r c.run
+
+ : No need to re-run a.test anymore, but c.test should be rerun,
+ : as it contained an XPASS. And this time, make it fail with
+ : an hard error.
+ # Use `echo' here, since Solaris 10 /bin/sh would try to optimize
+ # a `:' away after the first iteration, even if it is redirected.
+ echo dummy > c.err
+ env TEST_LOGS='a.log c.log' $MAKE -e recheck >stdout \
+ && { cat stdout; Exit 1; }
+ cat stdout
+ count_test_results total=1 pass=0 fail=0 xpass=0 xfail=0 skip=0 error=1
+ test ! -r a.run
+ test ! -r b.run
+ test -f c.run
+
+ rm -f *.run *.err
+
+ : c.test contained and hard error the last time, so it should be re-run.
+ : This time, make it pass
+ # Use `echo', not `:'; see comments above for why.
+ echo dummy > c.ok
+ env TESTS='c.test a.test' $MAKE -e recheck >stdout \
+ || { cat stdout; Exit 1; }
+ cat stdout
+ count_test_results total=1 pass=1 fail=0 xpass=0 xfail=0 skip=0 error=0
+ test ! -r a.run
+ test ! -r b.run
+ test -f c.run
+
+ rm -f *.run *.err *.ok
+
+ : Nothing should be rerun anymore, as all tests have been eventually
+ : succesful.
+ $MAKE recheck >stdout || { cat stdout; Exit 1; }
+ cat stdout
+ count_test_results total=0 pass=0 fail=0 xpass=0 xfail=0 skip=0 error=0
+ test ! -r a.run
+ test ! -r b.run
+ test ! -r c.run
+
+ cd $srcdir
+
+done
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Custom test drivers: check that we can easily support test protocols
+# that allow multiple testcases in a single test script. This test not
+# only checks implementation details in Automake's custom test drivers
+# support, but also serves as a "usability test" for our APIs.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cp "$testsrcdir"/trivial-test-driver . \
+ || fatal_ "failed to fetch auxiliary script trivial-test-driver"
+
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+TEST_EXTENSIONS = .t
+T_LOG_DRIVER = $(SHELL) $(srcdir)/trivial-test-driver
+
+TESTS = \
+ pass.t \
+ fail.t \
+ fail2.t \
+ pass-fail.t \
+ pass4-skip.t \
+ pass3-skip2-xfail.t \
+ pass-xpass-fail-xfail-skip-error.t
+END
+
+cat > pass.t << 'END'
+echo %% pass %%
+echo PASS: pass
+END
+
+cat > fail.t << 'END'
+echo %% fail %%
+echo FAIL: fail
+END
+
+cat > fail2.t << 'END'
+echo %% fail2 %%
+echo FAIL: stdout >&1
+echo FAIL: stderr >&2
+echo :PASS: this should be ignored
+END
+
+cat > pass-fail.t << 'END'
+echo %% pass-fail %%
+echo 'FAIL: this fails :-('
+echo 'some randome message'
+echo 'some randome warning' >&2
+echo 'PASS: this passes :-)'
+echo 'INFO: blah'
+echo 'WARNING: blah blah' >&2
+END
+
+cat > pass4-skip.t << 'END'
+echo %% pass4-skip %%
+echo PASS: on stdout >&1
+echo PASS: on stderr >&2
+echo PASS: 3
+echo PASS: 4
+echo SKIP: 1
+echo this FAIL: should be ignored
+echo FAIL as should this
+exit 99
+END
+
+cat > pass3-skip2-xfail.t << 'END'
+echo %% pass4-skip2-xfail %%
+echo 'PASS: -v'
+echo 'PASS: --verbose'
+echo 'SKIP: Oops, unsupported system.'
+echo 'PASS: -#-#-#-'
+cp || echo "SKIP: cp cannot read users' mind" >&2
+mv || echo "XFAIL: mv cannot read users' mind yet"
+exit 127
+END
+
+cat > pass-xpass-fail-xfail-skip-error.t << 'END'
+echo PASS:
+echo FAIL:
+echo XFAIL:
+echo XPASS:
+echo SKIP:
+echo ERROR:
+echo %% pass-xpass-fail-xfail-skip-error %%
+END
+
+chmod a+x *.t
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE
+
+for vpath in : false; do
+ if $vpath; then
+ mkdir build
+ cd build
+ srcdir=..
+ else
+ srcdir=.
+ fi
+
+ $srcdir/configure
+
+ $MAKE check >stdout && { cat stdout; cat test-suite.log; Exit 1; }
+ cat stdout
+ cat test-suite.log
+ # Couple of sanity checks. These might need to be updated if the
+ # `trivial-test-driver' script is changed.
+ $FGREP INVALID.NAME stdout test-suite.log && Exit 1
+ test -f BAD.LOG && Exit 1
+ test -f BAD.TRS && Exit 1
+ # These log files must all have been created by the testsuite.
+ cat pass.log
+ cat fail.log
+ cat fail2.log
+ cat pass-fail.log
+ cat pass4-skip.log
+ cat pass3-skip2-xfail.log
+ cat pass-xpass-fail-xfail-skip-error.log
+
+ count_test_results total=23 pass=10 fail=5 skip=4 xfail=2 xpass=1 error=1
+
+ tst=pass-xpass-fail-xfail-skip-error
+ grep "^PASS: $tst\.t, testcase 1" stdout
+ grep "^FAIL: $tst\.t, testcase 2" stdout
+ grep "^XFAIL: $tst\.t, testcase 3" stdout
+ grep "^XPASS: $tst\.t, testcase 4" stdout
+ grep "^SKIP: $tst\.t, testcase 5" stdout
+ grep "^ERROR: $tst\.t, testcase 6" stdout
+
+ # Check that the content of, and only of, the test logs with at least
+ # one failing test case has been copied into `test-suite.log'. Note
+ # that test logs containing skipped or xfailed test cases are *not*
+ # copied into `test-suite.log' -- a behaviour that deliberately differs
+ # from the one of the built-in Automake test drivers.
+ grep '%%' test-suite.log # For debugging.
+ grep '%% fail %%' test-suite.log
+ grep '%% fail2 %%' test-suite.log
+ grep '%% pass-fail %%' test-suite.log
+ grep '%% pass-xpass-fail-xfail-skip-error %%' test-suite.log
+ test `grep -c '%% ' test-suite.log` -eq 4
+
+ TESTS='pass.t pass3-skip2-xfail.t' $MAKE -e check >stdout \
+ || { cat stdout; cat test-suite.log; Exit 1; }
+ cat test-suite.log
+ cat stdout
+ count_test_results total=7 pass=4 fail=0 skip=2 xfail=1 xpass=0 error=0
+
+ cd $srcdir
+
+done
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check that auxiliary script 'test-driver' doesn't get needlessly
+# installed or referenced if it's not used, i.e., if the user has
+# defined his own `*LOG_DRIVER' variables.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.in <<'END'
+AC_PROG_CC
+AC_CONFIG_FILES([sub1/Makefile sub2/Makefile])
+AC_SUBST([X_LOG_DRIVER], [none])
+AC_OUTPUT
+END
+
+mkdir sub1 sub2
+
+cat > Makefile.am <<'END'
+SUBDIRS = sub1 sub2
+LOG_DRIVER =
+TEST_LOG_DRIVER = :
+TESTS = foo bar.test
+END
+
+cat > sub1/Makefile.am <<'END'
+TEST_EXTENSIONS = .x .sh .pl
+SH_LOG_DRIVER = dummy1
+PL_LOG_DRIVER = dummy2
+TESTS = a.pl b.sh c.x
+END
+
+cat > sub2/Makefile.am <<'END'
+TEST_EXTENSIONS = .bar
+LOG_DRIVER = x
+BAR_LOG_DRIVER = y
+TESTS = 1 2.bar 3.test 4.t 5.tt $(check_PROGRAMS)
+check_PROGRAMS = p1 p2$(EXEEXT) p3.bar p4.suf
+END
+
+$ACLOCAL
+
+for opts in '' '--add-missing' '-a -c'; do
+ $AUTOMAKE $opts
+ $FGREP test-driver Makefile.in sub[12]/Makefile.in && Exit 1
+ find . | $FGREP test-driver && Exit 1
+done
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check that custom test drivers does not need to produce sensible
+# reStructuredText output in the test logs. This might be legitimate
+# for drivers that are not interested to support the .log -> HTML
+# conversion offered by Automake.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+TEST_LOG_DRIVER = ./no-rst
+TESTS = foo.test
+END
+
+: > foo.test
+
+cat > no-rst <<'END'
+#! /bin/sh
+echo ':test-result: SKIP' > foo.trs
+echo ':copy-in-global-log: yes' >> foo.trs
+# The genereted log file is deliberately syntactically invalid
+# reStructuredText.
+cat > foo.log <<'EOF'
+SKIP: FooBar
+=============
+
+--------------
+ dummy title
+EOF
+END
+chmod a+x no-rst
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE
+
+./configure
+VERBOSE=yes $MAKE check
+cat foo.log
+cat test-suite.log
+$FGREP 'dummy title' test-suite.log
+
+# Sanity check: trying to produce HTML output should fail.
+$MAKE check-html >output 2>&1 && { cat output; Exit 1; }
+cat output
+# The second grep is to avoid spurious failures on systems without
+# a working `rst2html' program.
+$FGREP SEVERE output || $FGREP 'cannot find rst2html' output
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Custom test drivers: "abstract" XFAIL_TESTS support.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.in <<'END'
+AC_SUBST([nihil], [])
+AC_SUBST([ac_xfail_tests], ['x5.test x6$(test_suffix)'])
+AC_CONFIG_FILES([sub1/Makefile sub2/Makefile])
+AC_OUTPUT
+END
+
+cat > Makefile.am <<'END'
+SUBDIRS = . sub1 sub2
+TEST_LOG_DRIVER = $(srcdir)/td
+TESTS = pass.test xfail.test
+XFAIL_TESTS = xfail.test
+END
+
+mkdir sub1 sub2
+
+cat > sub1/Makefile.am <<END
+empty =
+
+TEST_LOG_DRIVER = \$(top_srcdir)/td
+
+# XFAIL_TESTS should gracefully handle TAB characters, and multiple
+# whitespaces.
+XFAIL_TESTS =\$(empty)${tab}x1.test x2.test${tab}x3.test${tab}\
+x4.test ${tab} x5.test x6.test${tab}\$(empty)
+
+TESTS = pass.test x1.test x2.test x3.test x4.test x5.test x6.test
+END
+
+cat > sub2/Makefile.am <<'END'
+AUTOMAKE_OPTIONS = -Wno-portability-recursive
+
+TEST_LOG_DRIVER = $(srcdir)/../td
+
+# XFAIL_TESTS should gracefully AC_SUBST @substitution@ and
+# make variables indirections.
+an_xfail_test = x1.test
+test_suffix = .test
+v0 = x3.test
+v1 = v
+v2 = 0
+XFAIL_TESTS = $(an_xfail_test) x2.test @nihil@ x3${test_suffix}
+XFAIL_TESTS += $($(v1)$(v2)) x4.test @ac_xfail_tests@
+
+TESTS = pass.test x1.test x2.test x3.test x4.test x5.test x6.test
+END
+
+cat > pass.test <<'END'
+#!/bin/sh
+exit 0
+END
+
+cat > xfail.test <<'END'
+#!/bin/sh
+exit 1
+END
+
+chmod a+x pass.test xfail.test
+
+cp pass.test sub1/pass.test
+cp pass.test sub2/pass.test
+
+for i in 1 2 3 4 5 6; do
+ cp xfail.test sub1/x$i.test
+ cp xfail.test sub2/x$i.test
+done
+
+cat > td <<'END'
+#! /bin/sh
+set -e; set -u
+test_name=INVALID
+log_file=/dev/null
+trs_file=/dev/null
+expect_failure=no
+while test $# -gt 0; do
+ case $1 in
+ --test-name) test_name=$2; shift;;
+ --expect-failure) expect_failure=$2; shift;;
+ --log-file) log_file=$2; shift;;
+ --trs-file) trs_file=$2; shift;;
+ # Ignored.
+ --color-tests) shift;;
+ --enable-hard-errors) shift;;
+ # Explicitly terminate option list.
+ --) shift; break;;
+ # Shouldn't happen
+ *) echo "$0: invalid option/argument: '$1'" >&2; exit 2;;
+ esac
+ shift
+done
+st=0
+"$@" || st=$?
+case $st,$expect_failure in
+ 0,no)
+ echo "PASS: $test_name" | tee "$log_file"
+ echo ":test-result: PASS" > "$trs_file"
+ ;;
+ 1,no)
+ echo "FAIL: $test_name" | tee "$log_file"
+ echo ":test-result: FAIL" > "$trs_file"
+ ;;
+ 0,yes)
+ echo "XPASS: $test_name" | tee "$log_file"
+ echo ":test-result: XPASS" > "$trs_file"
+ ;;
+ 1,yes)
+ echo "XFAIL: $test_name" | tee "$log_file"
+ echo ":test-result: XFAIL" > "$trs_file"
+ ;;
+ *)
+ echo "INTERNAL ERROR" >&2
+ exit 99
+ ;;
+esac
+END
+chmod a+x td
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE
+
+./configure
+
+$MAKE check >stdout || { cat stdout; Exit 1; }
+cat stdout
+test `grep -c '^PASS:' stdout` -eq 3
+test `grep -c '^XFAIL:' stdout` -eq 13
+
+for dir in sub1 sub2; do
+ cd $dir
+ cp pass.test x1.test
+ cp x2.test pass.test
+ $MAKE check >stdout && { cat stdout; Exit 1; }
+ cat stdout
+ test "`cat pass.trs`" = ":test-result: FAIL"
+ test "`cat x1.trs`" = ":test-result: XPASS"
+ test "`cat x2.trs`" = ":test-result: XFAIL"
+ grep '^FAIL: pass\.test$' stdout
+ grep '^XPASS: x1\.test$' stdout
+ grep '^XFAIL: x2\.test$' stdout
+ count_test_results total=7 pass=0 xpass=1 fail=1 xfail=5 skip=0 error=0
+ cd ..
+done
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Custom test drivers: per-extension test drivers.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+# Automake shouldn't match the '/test' in 'sub/test' as '.test' suffix.
+TESTS = 1.chk 2.test 3 4.c.chk 5.suf sub/test
+
+TEST_EXTENSIONS = .chk .test
+
+CHK_LOG_DRIVER = ./chk-wrapper
+TEST_LOG_DRIVER = $(SHELL) $(srcdir)/test-wrapper
+LOG_DRIVER = noext-wrapper
+
+AM_CHK_LOG_DRIVER_FLAGS = --am-chk
+CHK_LOG_DRIVER_FLAGS = --chk
+AM_TEST_LOG_DRIVER_FLAGS = -am-test
+TEST_LOG_DRIVER_FLAGS = -test
+AM_LOG_DRIVER_FLAGS = am
+LOG_DRIVER_FLAGS = _
+END
+
+mkdir sub bin
+PATH=`pwd`/bin$PATH_SEPARATOR$PATH; export PATH
+
+cat > wrapper.skel <<'END'
+#! /bin/sh
+set -e
+
+me=`echo "$0" | sed 's,^.*/,,'`
+if test -z "$me"; then
+ echo "$0: cannot determine program name" >&2
+ exit 99
+fi
+
+am_log_wflags='@am_log_wflags@'
+log_wflags='@log_wflags@'
+
+test_name=INVALID
+log_file=BAD.log
+trs_file=BAD.trs
+extra_opts=
+while test $# -gt 0; do
+ case $1 in
+ --test-name) test_name=$2; shift;;
+ --log-file) log_file=$2; shift;;
+ --trs-file) trs_file=$2; shift;;
+ # Ignored.
+ --expect-failure) shift;;
+ --color-tests) shift;;
+ --enable-hard-errors) shift;;
+ # Remembered in the same order they're passed in.
+ $am_log_wflags|$log_wflags) extra_opts="$extra_opts $1";;
+ # Explicitly terminate option list.
+ --) shift; break;;
+ # Shouldn't happen
+ *) echo "$0: invalid option/argument: '$1'" >&2; exit 2;;
+ esac
+ shift
+done
+
+echo "$me" "$test_name" $extra_opts > "$log_file"
+: > "$trs_file"
+
+exec "$@"
+exit 127
+END
+
+sed -e 's|@am_log_wflags@|--am-chk|' \
+ -e 's|@log_wflags@|--chk|' \
+ < wrapper.skel > chk-wrapper
+
+sed -e 's|@am_log_wflags@|-am-test|' \
+ -e 's|@log_wflags@|-test|' \
+ < wrapper.skel > test-wrapper
+
+sed -e 's|@am_log_wflags@|am|' \
+ -e 's|@log_wflags@|_|' \
+ < wrapper.skel > bin/noext-wrapper
+
+# `test-wrapper' is deliberately not made executable.
+chmod a+x chk-wrapper bin/noext-wrapper
+
+# Not needed anymore.
+rm -f wrapper.skel
+
+cat > 1.chk << 'END'
+#! /bin/sh
+exit 0
+END
+chmod a+x 1.chk
+cp 1.chk 2.test
+cp 1.chk 3
+cp 1.chk 4.c.chk
+cp 1.chk 5.suf
+cp 1.chk sub/test
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE
+
+./configure
+$MAKE
+VERBOSE=yes $MAKE check
+ls -l . sub
+
+test ! -r BAD.log
+test ! -r BAD.trs
+
+echo 'chk-wrapper 1.chk --am-chk --chk' > 1.exp
+echo 'test-wrapper 2.test -am-test -test' > 2.exp
+echo 'noext-wrapper 3 am _' > 3.exp
+echo 'chk-wrapper 4.c.chk --am-chk --chk' > 4.c.exp
+echo 'noext-wrapper 5.suf am _' > 5.suf.exp
+echo 'noext-wrapper sub/test am _' > sub/test.exp
+
+st=0
+for x in 1 2 3 4.c 5.suf sub/test; do
+ cat $x.log
+ diff $x.exp $x.log || st=1
+done
+
+Exit $st
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Custom test drivers: what happens when a test driver fails? Well,
+# "make check" should at least fail too, and the test-suite.log
+# shouldn't be created. Unfortunately, we cannot truly control also
+# the (non-)creation of individual test logs, since those are expected
+# to be created by the drivers themselves, and an ill-behaved driver
+# (like our dummy one in this test) might leave around a test log even
+# in case of internal failures.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.in <<'END'
+AC_OUTPUT
+END
+
+cat > Makefile.am <<'END'
+TEST_LOG_DRIVER = ./oops
+TESTS = foo.test
+END
+
+cat > foo.test <<'END'
+#! /bin/sh
+exit 0
+END
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE
+
+./configure
+
+# The testsuite driver does not exist.
+$MAKE check && Exit 1
+test ! -f test-suite.log
+
+# The testsuite driver exists and create the test log files, but fails.
+
+cat > oops <<'END'
+#!/bin/sh
+: > foo.log
+echo 'Oops, I fail!' >&2
+exit 1
+END
+chmod a+x oops
+
+$MAKE check && Exit 1
+test ! -f test-suite.log
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Custom test drivers: check that the test name passed to the test
+# driver has any VPATH prefix stripped.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+ocwd=`pwd` || fatal_ "cannot get current working directory"
+
+mkdir src build
+mv install-sh missing configure.in src
+rm -f depcomp
+
+cd src
+
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+# The directories of the `bar.test' and `baz.test' tests are deliberately
+# called as the source directory, to verify that the VPATH-stripping code
+# doesn't get too easily confused.
+# The $(empty) are for eliciting VPATH rewrites on make implementations
+# that support it (e.g., Solaris make), to improve coverage.
+empty =
+TESTS = $(empty) foo.test src/bar.test ./src/baz.test $(empty)
+$(TESTS):
+TEST_LOG_DRIVER = $(srcdir)/checkstrip-driver
+EXTRA_DIST = checkstrip-driver
+END
+
+cat > checkstrip-driver <<'END'
+#! /bin/sh
+set -e
+while test $# -gt 0; do
+ case $1 in
+ --log-file) log_file=$2; shift;;
+ --trs-file) trs_file=$2; shift;;
+ --test-name) test_name=$2; shift;;
+ --expect-failure|--color-tests|--enable-hard-errors) shift;;
+ --) shift; break;;
+ *) echo "$0: invalid option/argument: '$1'" >&2; exit 2;;
+ esac
+ shift
+done
+echo "test name: $test_name" # For debugging.
+case $test_name in
+ foo.test|./foo.test|src/ba[rz].test|./src/ba[rz].test);;
+ *) exit 1;;
+esac
+echo dummy > "$log_file"
+echo dummy > "$trs_file"
+END
+chmod a+x checkstrip-driver
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE
+
+cd ..
+
+mkdir build1
+cd build1
+../src/configure
+# "$MAKE -n" is for debugging, should highlight any VPATH rewrite.
+$MAKE -n check || :
+$MAKE check
+cd ..
+
+mkdir build2
+cd build2
+"$ocwd"/src/configure
+# "$MAKE -n" is for debugging, should highlight any VPATH rewrite.
+$MAKE -n check || :
+$MAKE check
+cd ..
+
+cd src
+./configure
+$MAKE distcheck
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# parallel-tests support: the following are registered with `.SUFFIXES':
+# - .log
+# - .trs (used by files that store test results and metadata)
+# - .test if $(TEST_EXTENSIONS) is not defined
+# - stuff in $(TEST_EXTENSIONS) otherwise
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+: > Makefile.am
+
+cat > 1.am << 'END'
+TESTS =
+END
+
+cat > 2.am << 'END'
+TEST_EXTENSIONS = .SH .abcdef
+TESTS =
+END
+
+: > test-driver
+
+$ACLOCAL
+
+$AUTOMAKE 1
+$AUTOMAKE 2
+
+sed -e 's/$/ /' 1.in > mk.1
+sed -e 's/$/ /' 2.in > mk.2
+
+grep '^\.SUFFIXES:' mk.1
+grep '^\.SUFFIXES:' mk.2
+
+for suf in test log trs; do
+ grep "^\\.SUFFIXES:.* \\.$suf " mk.1
+done
+
+for suf in SH abcdef log trs; do
+ grep "^\\.SUFFIXES:.* \\.$suf " mk.2
+done
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2010, 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check that our concurrent test harness is not subject to spurious VPATH
+# rewrites.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+LOG_COMPILER = true
+TESTS = x0 foo bar baz x1
+EXTRA_DIST = $(TESTS)
+END
+
+: > x0
+: > foo
+: > bar
+: > baz
+: > x1
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+mkdir build
+cd build
+../configure
+$MAKE check
+test -f test-suite.log
+test -f foo.log
+test -f foo.trs
+test -f bar.log
+test -f bar.trs
+test -f baz.log
+test -f baz.trs
+# We don't want VPATH rewrites, really.
+$MAKE check -n | $EGREP '\.\./(foo|bar|baz)( |$)' && Exit 1
+$MAKE distcheck
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check parallel-tests features:
+# - log file creation
+# - log file removal
+# - stdout and stderr of a test script go in its log file
+# - TEST_SUITE_LOG redefinition, at either automake or make time
+# - VERBOSE environment variable support
+# Keep in sync with 'tap-log.test'.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.in <<END
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+TESTS = pass.test skip.test xfail.test fail.test xpass.test error.test
+XFAIL_TESTS = xpass.test xfail.test
+TEST_SUITE_LOG = global.log
+END
+
+# Custom markers, for use in grepping checks.
+cmarker=::: # comment marker
+pmarker=%%% # plain maker
+
+cat > pass.test <<END
+#! /bin/sh
+echo "$pmarker pass $pmarker" >&2
+echo "# $cmarker pass $cmarker" >&2
+exit 0
+END
+
+cat > skip.test <<END
+#! /bin/sh
+echo "$pmarker skip $pmarker"
+echo "# $cmarker skip $cmarker"
+exit 77
+END
+
+cat > xfail.test <<END
+#! /bin/sh
+echo "$pmarker xfail $pmarker" >&2
+echo "# $cmarker xfail $cmarker" >&2
+exit 1
+END
+
+cat > fail.test <<END
+#! /bin/sh
+echo 1..1
+echo "$pmarker fail $pmarker"
+echo "# $cmarker fail $cmarker"
+exit 1
+END
+
+cat > xpass.test <<END
+#! /bin/sh
+echo "$pmarker xpass $pmarker" >&2
+echo "# $cmarker xpass $cmarker" >&2
+exit 0
+END
+
+cat > error.test <<END
+#! /bin/sh
+echo "$pmarker error $pmarker"
+echo "# $cmarker error $cmarker"
+exit 99
+END
+
+chmod a+x *.test
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+./configure
+
+TEST_SUITE_LOG=my.log $MAKE -e check && Exit 1
+ls -l # For debugging.
+test ! -f test-suite.log
+test ! -f global.log
+test -f my.log
+st=0
+for result in pass fail xfail xpass skip error; do
+ cat $result.log # For debugging.
+ $FGREP "$pmarker $result $pmarker" $result.log || st=1
+ $FGREP "$cmarker $result $cmarker" $result.log || st=1
+done
+test $st -eq 0 || Exit 1
+cat my.log # For debugging.
+for result in xfail fail xpass skip error; do
+ cat $result.log # For debugging.
+ $FGREP "$pmarker $result $pmarker" my.log || st=1
+ $FGREP "$cmarker $result $cmarker" my.log || st=1
+done
+test `$FGREP -c "$pmarker" my.log` -eq 5
+test `$FGREP -c "$cmarker" my.log` -eq 5
+
+have_rst_section ()
+{
+ eqeq=`echo "$1" | sed 's/./=/g'`
+ # Assume $1 contains no RE metacharacters.
+ sed -n "/^$1$/,/^$eqeq$/p" $2 > got
+ (echo "$1" && echo "$eqeq") > exp
+ cat exp
+ cat got
+ diff exp got
+}
+
+# Passed test scripts shouldn't be mentioned in the global log.
+$EGREP ':.*[^x]pass' my.log && Exit 1
+# But failing (expectedly or not) and skipped ones should.
+have_rst_section 'SKIP: skip' my.log
+have_rst_section 'FAIL: fail' my.log
+have_rst_section 'XFAIL: xfail' my.log
+have_rst_section 'XPASS: xpass' my.log
+have_rst_section 'ERROR: error' my.log
+
+touch error2.log test-suite.log global.log
+TEST_SUITE_LOG=my.log $MAKE -e mostlyclean
+ls -l # For debugging.
+test ! -f my.log
+test ! -f pass.log
+test ! -f fail.log
+test ! -f xfail.log
+test ! -f xpass.log
+test ! -f skip.log
+test ! -f error.log
+# "make mostlyclean" shouldn't remove unrelated log files.
+test -f error2.log
+test -f test-suite.log
+test -f global.log
+
+rm -f *.log
+
+VERBOSE=yes $MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+cat global.log
+test ! -f my.log
+test ! -f test-suite.log
+# Check that VERBOSE causes the global testsuite log to be
+# emitted on stdout.
+out=`cat stdout`
+log=`cat global.log`
+case $out in *"$log"*) ;; *) Exit 1;; esac
+
+touch error2.log test-suite.log my.log
+$MAKE clean
+ls -l # For debugging.
+test ! -f global.log
+test ! -f pass.log
+test ! -f fail.log
+test ! -f xfail.log
+test ! -f xpass.log
+test ! -f skip.log
+test ! -f error.log
+# "make clean" shouldn't remove unrelated log files.
+test -f error2.log
+test -f test-suite.log
+test -f my.log
+
+rm -f *.log
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check parallel-tests features:
+# - $(TESTS_LOGS) can end up holding duplicated entries
+# Such behaviour can be useful for projects that want to verify that
+# different test scripts don't end up sharing the same log file.
+# (Automake itself is such a project, BTW ;-)
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.in <<END
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+TEST_EXTENSIONS = .sh .test .bar
+TESTS = foo.sh mu.test foo.test foo.bar
+.PHONY: verify
+verify:
+ test "`echo $(TEST_LOGS)`" = 'foo.log mu.log foo.log foo.log'
+END
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+./configure
+$MAKE verify
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Custom test drivers and parallel-tests harness: check the documented
+# semantics for deciding when the content of a test log file should be
+# copied in the global test-suite.log file. Currently, this is done
+# with the use of the reStructuredText field `:copy-in-global-log:' in
+# the associated `.trs' files.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+TEST_LOG_DRIVER = ./passthrough-driver
+TEST_LOG_COMPILER = $(SHELL) -e
+END
+
+cat > passthrough-driver <<'END'
+#!/bin/sh
+set -e; set -u;
+while test $# -gt 0; do
+ case $1 in
+ --log-file) log_file=$2; shift;;
+ --trs-file) trs_file=$2; shift;;
+ --test-name) test_name=$2; shift;;
+ --expect-failure|--color-tests|--enable-hard-errors) shift;;
+ --) shift; break;;
+ *) echo "$0: invalid option/argument: '$1'" >&2; exit 2;;
+ esac
+ shift
+done
+echo "$test_name: RUN"
+"$@" >$log_file 2>&1 5>$trs_file
+END
+chmod a+x passthrough-driver
+
+# The `:test-result:' and `:recheck:' fields and the first line of the
+# log file should be be irrelevant for the decision of whether a test
+# output is to be copied in the `test-suite.log'.
+
+cat > no-1.test <<END
+echo :test-result: SKIP >&5
+echo :copy-in-global-log: no >&5
+echo :test-result: FAIL >&5
+echo :test-result: XPASS >&5
+echo not seen 1
+END
+
+# In the last line, with leading and trailing whitespace in the value.
+cat > no-2.test <<END
+echo ":test-result: FAIL" >&5
+echo "not seen 2"
+echo ":recheck: yes" >&5
+echo ":copy-in-global-log:$tab $tab no $tab" >&5
+END
+
+for RES in XPASS FAIL XFAIL SKIP ERROR UNKNOWN; do
+ unindent > $RES.test <<END
+ echo :test-result: $RES >&5
+ echo :copy-in-global-log: no >&5
+ echo not seen $RES
+END
+done
+
+# In the first line, with no whitespace.
+cat > no-3.test <<END
+echo :copy-in-global-log:no >&5
+echo ":test-result: FAIL" >&5
+echo "not seen 3"
+END
+
+# Leading whitespace before the field.
+cat > no-4.test <<END
+echo ":test-result: FAIL" >&5
+echo " $tab $tab$tab :copy-in-global-log: no" >&5
+echo "not seen 4"
+END
+
+cat > yes-1.test <<END
+echo :test-result: PASS >&5
+echo :copy-in-global-log: yes >&5
+echo seen yes 1
+END
+
+# A lacking `:copy-in-global-log:' implies that the content of
+# the log file should be copied.
+cat > yes-2.test <<END
+echo :test-result: PASS >&5
+echo seen yes 2
+END
+
+# Three corner cases.
+
+cat > corn-1.test <<END
+echo seen corn 1
+echo ':copy-in-global-log:' >&5
+END
+
+cat > corn-2.test <<END
+echo seen corn 2
+echo '$tab $tab$tab' >&5
+END
+
+cat > corn-3.test <<'END'
+echo seen corn 31
+echo ':copy-in-global-log:#@%!' >&5
+echo seen corn 32
+END
+
+echo TESTS = *.test >> Makefile.am
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE
+
+./configure
+
+# We don't care about the exit status of "make check" here, that
+# should be checked in other tests.
+$MAKE check || :
+cat test-suite.log
+grep '^seen yes 1$' test-suite.log
+grep '^seen yes 2$' test-suite.log
+grep '^seen corn 1$' test-suite.log
+grep '^seen corn 2$' test-suite.log
+grep '^seen corn 31$' test-suite.log
+grep '^seen corn 32$' test-suite.log
+$FGREP 'not seen' test-suite.log && Exit 1
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Parallel testsuite harness: check APIs for the registering the
+# "global test result" in `*.trs' files, as documented in the automake
+# manual.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+TEST_EXTENSIONS = .test .x
+TEST_LOG_DRIVER = ./dummy-driver
+X_LOG_DRIVER = ./dummy-driver
+TESTS = foo.test zar-doz.test
+END
+
+cat > dummy-driver <<'END'
+#! /bin/sh
+set -e; set -u
+while test $# -gt 0; do
+ case $1 in
+ --log-file) log_file=$2; shift;;
+ --trs-file) trs_file=$2; shift;;
+ --test-name) test_name=$2; shift;;
+ --expect-failure|--color-tests|--enable-hard-errors) shift;;
+ --) shift; break;;
+ *) echo "$0: invalid option/argument: '$1'" >&2; exit 2;;
+ esac
+ shift
+done
+echo logloglog > $log_file
+cp $1 $trs_file
+END
+chmod a+x dummy-driver
+
+# Do this in a subroutine to avoid quoting problem in the backticked
+# command substitution below.
+get_escaped_line()
+{
+ sed -e 's,[$^/\\\.],\\&,g' -e 1q "$@"
+}
+
+have_result ()
+{
+ cat > exp; echo >> exp; echo logloglog >> exp
+ eline=`get_escaped_line exp`
+ sed -n -e "/^$eline$/,/^logloglog$/p" test-suite.log > got
+ cat exp; cat got
+ diff exp got
+}
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE
+
+./configure
+
+: Basic checks.
+
+echo :global-test-result: PASS > foo.test
+echo :global-test-result: ERROR > zar-doz.x
+
+$MAKE check
+cat test-suite.log
+
+have_result <<END
+PASS: foo
+=========
+END
+
+have_result <<END
+ERROR: zar-doz
+==============
+END
+
+: Try usage documented in the manual.
+
+echo :global-test-result: PASS/SKIP > foo.test
+echo :global-test-result: ALMOST PASSED > zar-doz.x
+
+$MAKE check
+cat test-suite.log
+
+have_result <<END
+PASS/SKIP: foo
+==============
+END
+
+have_result <<END
+ALMOST PASSED: zar-doz
+======================
+END
+
+: Fields ':test-result:' does not interfere with the global test result.
+
+cat > foo.test << 'END'
+:test-result: FAIL
+:global-test-result: PASS
+:test-result: ERROR
+END
+
+cat > zar-doz.x << 'END'
+:global-test-result: FAIL
+:test-result: SKIP
+:test-result: XFAIL
+END
+
+$MAKE check && Exit 1
+cat test-suite.log
+
+have_result <<END
+PASS: foo
+=========
+END
+
+have_result <<END
+FAIL: zar-doz
+=============
+END
+
+: What happens when ':global-test-result:' is absent.
+
+cat > foo.test << 'END'
+:test-result: PASS
+:test-result: ERROR
+END
+: > zar-doz.x
+
+$MAKE check && Exit 1
+cat test-suite.log
+
+have_result <<END
+RUN: foo
+========
+END
+
+have_result <<END
+RUN: zar-doz
+============
+END
+
+# Leading and trailing whitespace gets eaten/normalized.
+
+echo ":global-test-result:SKIP${tab} ${tab}${tab}" > foo.test
+echo ":global-test-result:${tab} ${tab}XFAIL ${tab} " > zar-doz.x
+
+$MAKE check
+cat test-suite.log
+
+have_result <<END
+SKIP: foo
+=========
+END
+
+have_result <<END
+XFAIL: zar-doz
+==============
+END
+
+# Whitespaces before and after `:global-test-result:' are handled OK.
+
+echo " $tab:global-test-result:PASS" > foo.test
+echo "${tab}${tab}:global-test-result:${tab} ${tab}SKIP" > zar-doz.x
+
+$MAKE check
+cat test-suite.log
+
+have_result <<END
+PASS: foo
+=========
+END
+
+have_result <<END
+SKIP: zar-doz
+=============
+END
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Test the "make recheck" semantics for custom test drivers, as documented
+# in the Automake manual.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+LOG_DRIVER = ./dummy-driver
+TEST_EXTENSIONS =
+TESTS =
+END
+
+#
+# Tests to be re-run by "make recheck"
+#
+
+: > y-1
+echo "foo bar" > y-2
+echo ":recheck:" > y-3
+echo ":recheck:yes" > y-4
+echo ":recheck: who cares" > y-5
+echo ":recheck: $tab y" > y-6
+echo ":recheck: yeah!$tab$tab " > y-7
+cat > y-10 <<END
+:foo:
+:recheck: ???
+END
+cat > y-11 <<END
+:recheck: YES
+:foo:
+END
+cat > y-12 <<END
+foo
+:recheck:yes
+
+bar
+zardoz
+END
+
+echo " $tab $tab$tab :recheck: yes" > y-8
+
+# The :test-result: fields and the fist line of the log should be
+# irrelevant for the decision of whether "make recheck" should or
+# should not re-run a test.
+
+echo ":test-result: PASS" > y-100
+
+echo "PASS: y-101"
+
+cat > y-102 <<END
+PASS: y-102
+===========
+
+:test-result: PASS
+END
+
+#
+# Tests *not* to be re-run by "make recheck"
+#
+
+echo ":recheck:no" > n-1
+echo ":recheck: no " > n-2
+echo ":recheck: $tab no" > n-3
+echo ":recheck: no $tab$tab " > n-4
+cat > n-5 <<END
+:foo:
+:recheck:no
+END
+cat > n-6 <<END
+:recheck: no
+:foo:
+END
+cat > n-7 <<END
+foo
+:recheck: no$tab$tab
+
+bar
+zardoz
+END
+
+echo " $tab $tab$tab :recheck: no" > n-8
+
+# The :test-result: fields should be irrelevant for the decision of
+# whether "make recheck" should or should not re-run a test.
+cat > n-100 <<END
+:test-result: FAIL
+:test-result: XPASS
+:test-result: ERROR
+:test-result: UNKNOWN
+:recheck: no
+END
+
+rechecked=`echo y-[0-9]*`
+
+for t in [yn]-[0-9]*; do echo $t; done \
+ | sed 's/.*/TESTS += &/' >> Makefile.am
+
+cat Makefile.am # For debugging.
+
+cat > dummy-driver <<'END'
+#!/bin/sh
+set -e; set -u
+while test $# -gt 0; do
+ case $1 in
+ --log-file) log_file=$2; shift;;
+ --trs-file) trs_file=$2; shift;;
+ --test-name) test_name=$2; shift;;
+ --expect-failure|--color-tests|--enable-hard-errors) shift;;
+ --) shift; break;;
+ *) echo "$0: invalid option/argument: '$1'" >&2; exit 2;;
+ esac
+ shift
+done
+: > $test_name.run
+: > $log_file
+cp $1 $trs_file
+END
+chmod a+x dummy-driver
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE
+
+./configure
+
+# The ':test-result:' fields should be ignored by "make recheck",
+# but should cause the testsuite report to detect errors.
+$MAKE check && Exit 1
+ls -l
+for t in $tests; do test -f $t.run; done
+rm -f *.run
+
+# But now the tests that actually get re-run have only ':test-result:'
+# fields indicating success, so "make recheck" must pass. Still, the
+# next "make recheck" call should still re-run the same set of tests.
+for iteration in 1 2; do
+ $MAKE recheck
+ ls -l
+ for t in $rechecked; do test -f $t.run; done
+ find . -name 'n-*.run' | grep . && Exit 1
+done
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Parallel testsuite harness: check APIs for the registering of test
+# results in `*.trs' files, as documented in the automake manual.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+TEST_LOG_DRIVER = ./dummy-driver
+TESTS = foo.test bar.test
+END
+
+cat > dummy-driver <<'END'
+#! /bin/sh
+set -e; set -u
+while test $# -gt 0; do
+ case $1 in
+ --log-file) log_file=$2; shift;;
+ --trs-file) trs_file=$2; shift;;
+ --test-name) test_name=$2; shift;;
+ --expect-failure|--color-tests|--enable-hard-errors) shift;;
+ --) shift; break;;
+ *) echo "$0: invalid option/argument: '$1'" >&2; exit 2;;
+ esac
+ shift
+done
+: > $log_file
+cp $1 $trs_file
+END
+chmod a+x dummy-driver
+
+mk_check ()
+{
+ st=0
+ $MAKE check >stdout || st=$?
+ cat stdout
+ # Our dummy driver make no testsuite progress report.
+ grep ': .*\.test' stdout && Exit 1
+ # Nor it writes to the log files.
+ test -s foo.log && Exit 1
+ test -s bar.log && Exit 1
+ return $st
+}
+
+# This must be different from the one defined in `test/defs', as that
+# assumes that the driver does proper testsuite progress reporting.
+count_test_results ()
+{
+ total=ERR pass=ERR fail=ERR xpass=ERR xfail=ERR skip=ERR error=ERR
+ eval "$@"
+ st=0
+ grep "^# TOTAL: *$total$" stdout || rc=1
+ grep "^# PASS: *$pass$" stdout || rc=1
+ grep "^# XFAIL: *$xfail$" stdout || rc=1
+ grep "^# SKIP: *$skip$" stdout || rc=1
+ grep "^# FAIL: *$fail$" stdout || rc=1
+ grep "^# XPASS: *$xpass$" stdout || rc=1
+ grep "^# ERROR: *$error$" stdout || rc=1
+ test $st -eq 0 || Exit 1
+}
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE
+
+./configure
+
+# Basic checks. Also that that `:global-test-result:' fields and
+# "old-style" directives with format "RESULT: test-name" are now ignored.
+
+: > foo.test
+echo blah blah blah > bar.test
+mk_check
+count_test_results total=0 pass=0 fail=0 xpass=0 xfail=0 skip=0 error=0
+
+cat > foo.test <<END
+:test-global-result: PASS
+:test-result: FAIL
+END
+cat > bar.test <<END
+:test-result: SKIP
+:test-global-result: ERROR
+END
+mk_check && Exit 1
+count_test_results total=2 pass=0 fail=1 xpass=0 xfail=0 skip=1 error=0
+
+cat > foo.test <<END
+FAIL: foo.test
+:test-result: PASS
+:test-global-result: XPASS
+END
+echo ERROR: bar.test > bar.test
+mk_check
+count_test_results total=1 pass=1 fail=0 xpass=0 xfail=0 skip=0 error=0
+
+cat > foo.test <<END
+:test-global-result: SKIP
+:test-result: FAIL
+END
+cat > bar.test <<END
+:test-global-result: PASS
+END
+mk_check && Exit 1
+count_test_results total=1 pass=0 fail=1 xpass=0 xfail=0 skip=0 error=0
+
+cat > foo.test <<END
+:test-result: XFAIL
+:test-result: PASS
+:test-result: SKIP
+END
+cat > bar.test <<END
+:test-result: SKIP
+:test-result: PASS
+:test-result: SKIP
+:test-result: PASS
+:test-result: PASS
+END
+mk_check
+count_test_results total=8 pass=4 fail=0 xpass=0 xfail=1 skip=3 error=0
+
+# Check that all results expected to be supported are *really* supported.
+
+cat > foo.test <<END
+:test-result: PASS
+:test-result: SKIP
+:test-result: XFAIL
+:test-result: FAIL
+:test-result: XPASS
+:test-result: ERROR
+END
+: > bar.test
+mk_check && Exit 1
+count_test_results total=6 pass=1 fail=1 xpass=1 xfail=1 skip=1 error=1
+
+cp foo.test bar.test
+mk_check && Exit 1
+count_test_results total=12 pass=2 fail=2 xpass=2 xfail=2 skip=2 error=2
+
+# Check that we are liberal w.r.t. whitespace use.
+
+: > foo.test
+: > bar.test
+for RESULT in PASS FAIL XPASS XFAIL SKIP ERROR; do
+ sed -e 's/^ *//' -e 's/|//g' >> foo.test <<END
+ |:test-result:$RESULT|
+ |:test-result: $tab $RESULT|
+ |:test-result:$RESULT $tab|
+ |:test-result:$tab$tab $RESULT$tab $tab |
+END
+ echo " $tab $tab$tab :test-result: $RESULT" >> bar.test
+done
+cat foo.test # For debugging.
+cat bar.test # Likewise.
+mk_check && Exit 1
+count_test_results total=30 pass=5 fail=5 xpass=5 xfail=5 skip=5 error=5
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# parallel-tests:
+# - non-existent scripts listed in TESTS get diagnosed
+# See also related test 'test-missing2.test'.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+TESTS = ok.test zardoz.test
+TEST_LOG_COMPILER = true
+END
+
+: > ok.test
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+./configure
+
+$MAKE check >output 2>&1 && { cat output; Exit 1; }
+cat output
+test -f ok.log
+grep '^PASS: ok\.test' output
+$FGREP 'zardoz.log' output
+test ! -f test-suite.log
+
+TESTS='zardoz2.test' $MAKE -e check >output 2>&1 \
+ && { cat output; Exit 1; }
+cat output
+$FGREP 'zardoz2.log' output
+test ! -f test-suite.log
+
+TEST_LOGS='zardoz3.log' $MAKE -e check >output 2>&1 \
+ && { cat output; Exit 1; }
+cat output
+$FGREP 'zardoz3.log' output
+test ! -f test-suite.log
+
+# The errors should persist even after `test-suite.log'
+# has been created.
+
+: > zardoz.test
+$MAKE check
+rm -f zardoz.test
+
+$MAKE check >output 2>&1 && { cat output; Exit 1; }
+cat output
+$FGREP 'zardoz.log' output
+test ! -f test-suite.log
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# parallel-tests:
+# - non-existent scripts listed in TESTS get diagnosed, even when
+# all the $(TEST_LOGS) have a dummy dependency.
+# See also related test 'test-missing.test'.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+TESTS = foobar1.test foobar2.test
+$(TEST_LOGS):
+END
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+./configure
+
+$MAKE foobar1.log foobar2.log || Exit 99
+test ! -f foobar1.log || Exit 99
+test ! -f foobar1.trs || Exit 99
+test ! -f foobar2.log || Exit 99
+test ! -f foobar2.trs || Exit 99
+
+$MAKE check >output 2>&1 && { cat output; Exit 1; }
+cat output
+grep 'test-suite\.log.*foobar1\.log' output
+grep 'test-suite\.log.*foobar1\.trs' output
+grep 'test-suite\.log.*foobar2\.log' output
+grep 'test-suite\.log.*foobar2\.trs' output
+test ! -f test-suite.log
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check parallel harness features:
+# - creation and removal of `.trs' auxiliary files
+# - check some internals regarding the use of `.trs' files.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.in <<END
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+TEST_EXTENSIONS = .sh .test
+TESTS = foo.test bar.sh sub/zardoz.test
+TEST_LOG_COMPILER = $(SHELL)
+SH_LOG_COMPILER = $(SHELL)
+
+## Used to check some internal details. And yes, the quotes around
+## `$bases' are deliberate: they check for whitespace normalization.
+tb:
+ $(am__set_TESTS_bases); echo "$$bases" > $@
+END
+
+cat > foo.test << 'END'
+#! /bin/sh
+exit $FOO_STATUS
+END
+: > bar.sh
+mkdir sub
+: > sub/zardoz.test
+
+FOO_STATUS=0; export FOO_STATUS
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+
+#
+# Check some internal details first.
+#
+
+for vpath in : false; do
+
+ if $vpath; then
+ srcdir=..
+ mkdir build
+ cd build
+ else
+ srcdir=.
+ fi
+
+ $srcdir/configure
+
+ $MAKE tb
+ test x"`cat tb`" = x"foo bar sub/zardoz"
+ rm -f tb
+ # Please don't change the order of the stuff in TESTS, below.
+ TESTS='foo.test foo2.sh foo-log foolog.test a.log.b.sh 0.exe' $MAKE -e tb
+ test x"`cat tb`" = x"foo foo2 foo-log foolog a.log.b 0.exe"
+ rm -f tb
+
+ cd $srcdir
+
+done
+
+#
+# The `test-suite.stamp' file and the `.trs' files get created by
+# "make check" and removed by "make clean" and "make mostlyclean".
+#
+
+: > unrelated.trs
+: > sub/foo.trs
+
+$MAKE check
+test -f foo.trs
+test -f bar.trs
+test -f sub/zardoz.trs
+$MAKE clean
+test ! -f foo.trs
+test ! -f bar.trs
+test ! -f sub/zardoz.trs
+# Unrelated `.trs' files shouldn't be removed.
+test -f unrelated.trs
+test -f sub/foo.trs
+
+# The files should be properly created in case of testsuite failure too.
+FOO_STATUS=1 $MAKE check && Exit 1
+test -f foo.trs
+test -f bar.trs
+test -f sub/zardoz.trs
+$MAKE mostlyclean
+test ! -f foo.trs
+test ! -f bar.trs
+test ! -f sub/zardoz.trs
+# Unrelated `.trs' files shouldn't be removed.
+test -f unrelated.trs
+test -f sub/foo.trs
+
+#
+# Try with a subset of TESTS.
+#
+
+TESTS=foo.test $MAKE -e check
+test -f foo.trs
+test ! -f bar.trs
+test ! -f sub/zardoz.trs
+$MAKE clean
+test ! -f foo.trs
+TESTS='foo.test bar.sh' $MAKE -e check
+test -f foo.trs
+test -f bar.trs
+test ! -f sub/zardoz.trs
+# "make clean" shouldn't remove `.trs' files for tests not in $(TESTS).
+TESTS=bar.sh $MAKE -e clean
+test -f foo.trs
+test ! -f bar.trs
+
+$MAKE clean
+
+#
+# Try with a subset of TEST_LOGS.
+#
+
+TEST_LOGS=sub/zardoz.log $MAKE -e check
+test ! -f foo.trs
+test ! -f bar.trs
+test -f sub/zardoz.trs
+$MAKE clean
+test ! -f sub/zardoz.trs
+TEST_LOGS='foo.log bar.log' $MAKE -e check
+test -f foo.trs
+test -f bar.trs
+test ! -f sub/zardoz.trs
+# "make clean" shouldn't remove `.trs' files for tests whose log
+# is not in $(TEST_LOGS).
+TEST_LOGS=foo.log $MAKE -e clean
+test ! -f foo.trs
+test -f bar.trs
+test ! -f sub/zardoz.trs
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check parallel harness features:
+# - recovery from deleted `.trs' files, in various scenarios
+# This test is complex and tricky, but that's acceptable since we are
+# testing semantics that are potentially complex and tricky.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.in <<END
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+TESTS = foo.test bar.test baz.test
+TEST_LOG_COMPILER = $(SHELL)
+END
+
+# FIXME: creative quoting below to please maintainer-check.
+echo exit '$TEST_STATUS' > foo.test
+echo exit '$TEST_STATUS' > bar.test
+: > baz.test
+
+TEST_STATUS=0; export TEST_STATUS
+
+# Slower and possible overkill in some situations, but also clearer
+# and safer.
+update_stamp () { $sleep && touch stamp && $sleep; }
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+./configure
+
+: Create the required log files.
+$MAKE check
+
+: Recreate by hand.
+rm -f foo.trs bar.trs baz.trs
+$MAKE foo.trs
+test -f foo.trs
+test ! -f bar.trs
+test ! -f baz.trs
+
+: Recreate by hand, several at the same time.
+rm -f foo.trs bar.trs baz.trs
+$MAKE foo.trs bar.trs
+test -f foo.trs
+test -f bar.trs
+test ! -f baz.trs
+
+: Recreate by hand, with a failing test.
+rm -f foo.trs bar.trs
+TEST_STATUS=1 $MAKE bar.trs baz.trs >stdout || { cat stdout; Exit 1; }
+cat stdout
+test ! -f foo.trs
+test -f bar.trs
+test -f baz.trs
+grep '^FAIL: bar\.test' stdout
+$EGREP '^(baz|foo)\.test' stdout && Exit 1
+
+: Recreate with a sweeping "make check", and ensure that also up-to-date
+: '.trs' files are remade.
+update_stamp
+rm -f foo.trs bar.trs
+$MAKE check
+test -f foo.trs
+test -f bar.trs
+is_newest baz.trs stamp
+
+: Recreate with a sweeping "make check" with failing tests. Again,
+: ensure that also up-to-date '.trs' files are remade -- this time we
+: grep the "make check" output verify that.
+rm -f foo.trs bar.trs
+TEST_STATUS=1 $MAKE check >stdout && { cat stdout; Exit 1; }
+test -f foo.trs
+test -f bar.trs
+grep '^FAIL: foo\.test' stdout
+grep '^FAIL: bar\.test' stdout
+grep '^PASS: baz\.test' stdout
+
+: Recreate with a "make check" with redefined TESTS.
+rm -f foo.trs bar.trs baz.trs
+TESTS=foo.test $MAKE -e check
+test -f foo.trs
+test ! -f bar.trs
+test ! -f baz.trs
+
+: Recreate with a "make check" with redefined TEST_LOGS.
+rm -f foo.trs bar.trs baz.trs
+TEST_LOGS=bar.log $MAKE -e check
+test ! -f foo.trs
+test -f bar.trs
+test ! -f baz.trs
+
+: Interactions with "make recheck" are OK.
+rm -f foo.trs bar.trs baz.log baz.trs
+$MAKE recheck >stdout || { cat stdout; Exit 1; }
+cat stdout
+test -f foo.trs
+test -f bar.trs
+test ! -f baz.trs
+test ! -f baz.log
+grep '^PASS: foo\.test' stdout
+grep '^PASS: bar\.test' stdout
+grep 'baz\.test' stdout && Exit 1
+count_test_results total=2 pass=2 fail=0 xpass=0 xfail=0 skip=0 error=0
+
+: Setup for the next check.
+$MAKE check
+test -f foo.trs
+test -f bar.trs
+test -f baz.trs
+
+: Recreate by remaking the global test log, and ensure that up-to-date
+: '.trs' files are *not* remade.
+update_stamp
+rm -f foo.trs bar.trs test-suite.log
+$MAKE test-suite.log >stdout || { cat stdout; Exit 1; }
+cat stdout
+grep '^PASS: foo\.test' stdout
+grep '^PASS: bar\.test' stdout
+grep 'baz\.test' stdout && Exit 1
+stat *.trs *.log stamp || : # For debugging.
+# Check that make has updated what it needed to, but no more.
+test -f foo.trs
+test -f bar.trs
+is_newest stamp baz.trs
+is_newest test-suite.log foo.trs bar.trs
+
+: Setup for the next check.
+$MAKE check
+test -f foo.trs
+test -f bar.trs
+test -f baz.trs
+
+: Interactions with lazy test reruns are OK.
+rm -f foo.trs
+update_stamp
+touch bar.test
+RECHECK_LOGS= $MAKE -e check >stdout || { cat stdout; Exit 1; }
+cat stdout
+# Check that make has updated what it needed to, but no more.
+test -f foo.trs
+is_newest bar.trs bar.test
+is_newest stamp baz.trs
+grep '^PASS: foo\.test' stdout
+grep '^PASS: bar\.test' stdout
+grep 'baz\.test' stdout && Exit 1
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check parallel harness features:
+# - recovery from unreadable `.trs' files, in various scenarios
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.in <<END
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+TESTS = foo.test bar.test
+TEST_LOG_COMPILER = true
+END
+
+: > foo.test
+: > bar.test
+
+TEST_STATUS=0; export TEST_STATUS
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+./configure
+
+: > t
+chmod a-r t
+test ! -r t || Exit 77
+rm -f t
+
+: Create the required log files.
+$MAKE check
+test -f foo.trs
+test -f bar.trs
+
+: Recreate with a sweeping "make check".
+chmod a-r bar.trs
+$MAKE check
+test -f foo.trs
+test -r foo.trs
+test -f bar.trs
+test -r bar.trs
+
+: Again, this time with one .trs file missing and the other
+: one unreadable.
+rm -f foo.trs
+chmod a-r bar.trs
+$MAKE check
+test -f foo.trs
+test -r foo.trs
+test -f bar.trs
+test -r bar.trs
+
+: Again, but using "make recheck" this time.
+rm -f foo.trs
+chmod a-r bar.trs
+$MAKE recheck >stdout || { cat stdout; Exit 1; }
+cat stdout
+test -f foo.trs
+test -r foo.trs
+test -f bar.trs
+test -r bar.trs
+grep '^PASS: foo\.test' stdout
+grep '^PASS: bar\.test' stdout
+
+: More complex interactions with "make recheck" are OK.
+chmod a-r bar.log bar.trs
+$MAKE recheck >stdout || { cat stdout; Exit 1; }
+cat stdout
+test -f bar.trs
+test -r bar.trs
+grep '^PASS: bar\.test' stdout
+grep 'foo\.test' stdout && Exit 1
+count_test_results total=1 pass=1 fail=0 xpass=0 xfail=0 skip=0 error=0
+
+: Recreate by remaking the global test log.
+chmod a-r foo.trs
+rm -f test-suite.log
+$MAKE test-suite.log >stdout || { cat stdout; Exit 1; }
+cat stdout
+test -f foo.trs
+test -r foo.trs
+grep '^PASS: foo\.test' stdout
+grep 'bar\.test' stdout && Exit 1
+# Also test that have only run before should be counted in the final
+# testsuite summary.
+grep '^# TOTAL: *2$' stdout
+
+: Setup for the next check.
+: > baz.test
+sed 's/^TESTS =.*/& baz.test/' Makefile > t
+diff t Makefile && Exit 99
+mv -f t Makefile
+$MAKE check
+test -f foo.trs
+test -f bar.trs
+test -f baz.trs
+
+: Interactions with lazy test reruns are OK.
+chmod a-r foo.trs
+$sleep
+touch stamp
+$sleep
+touch bar.test
+RECHECK_LOGS= $MAKE -e check >stdout || { cat stdout; Exit 1; }
+cat stdout
+test -r foo.trs
+is_newest bar.trs bar.test
+grep '^PASS: foo\.test' stdout
+grep '^PASS: bar\.test' stdout
+grep 'baz\.test' stdout && Exit 1
+# Also test that have only run before should be counted in the final
+# testsuite summary.
+grep '^# TOTAL: *3$' stdout
+
+:
./configure
-$MAKE -n check # For debugging.
+$MAKE -n check || : # For debugging.
VERBOSE=yes $MAKE check
cat a.log
cat b.log
parallel_tests=no
. ./defs || Exit 1
-set -e
-
cat >> configure.in <<END
AC_SUBST([PERL], ['$PERL'])
AC_OUTPUT
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check the testsuite summary with the parallel-tests harness. This
+# script is meant to be sourced by other test script, so that it can
+# be used to check different scenarios (colorized and non-colorized
+# testsuite output, packages with and without bug-report addresses,
+# testsuites in subdirectories, ...)
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+case $use_colors in
+ yes)
+ AM_COLOR_TESTS=always; export AM_COLOR_TESTS
+ TERM=ansi; export TERM
+ am_opts='parallel-tests color-tests'
+ ;;
+ no)
+ am_opts='parallel-tests'
+ ;;
+ *)
+ fatal_ "invalid use_colors='$use_colors'";;
+esac
+
+cat > configure.in <<END
+AC_INIT([GNU AutoFoo], [7.1], [bug-automake@gnu.org])
+AM_INIT_AUTOMAKE([$am_opts])
+AC_CONFIG_FILES([Makefile])
+AC_OUTPUT
+END
+
+cat > Makefile.am <<'END'
+TEST_EXTENSIONS = .t
+T_LOG_COMPILER = $(SHELL) $(srcdir)/log-compiler
+## Will be defined at runtime.
+TESTS =
+$(TESTS):
+END
+
+cat > log-compiler <<'END'
+#!/bin/sh
+case "$1" in
+ pass*|*/pass*|xpass*|*/xpass*) exit 0;;
+ fail*|*/fail*|xfail*|*/xfail*) exit 1;;
+ skip*|*/skip*) exit 77;;
+ error*|/error*) exit 99;;
+ copy*|*/copy*) cat "$1";;
+ *) exit 99;;
+esac
+END
+
+# Quite complexish, but allow the tests in client scripts to be written
+# in a "data-driven fashion".
+do_check ()
+{
+ cat > summary.exp
+ expect_failure=false
+ xfail_tests=''
+ tests="TESTS='$*'"
+ for t in $*; do
+ case $t in fail*|xpass*|error*) expect_failure=:;; esac
+ case $t in xfail*|xpass*) xfail_tests="$xfail_tests $t";; esac
+ done
+ test -z "$xfail_tests" || xfail_tests="XFAIL_TESTS='$xfail_tests'"
+ st=0
+ eval "env $tests $xfail_tests \$MAKE -e check > stdout || st=\$?"
+ cat stdout
+ if $expect_failure; then
+ test $st -gt 0 || Exit 1
+ else
+ test $st -eq 0 || Exit 1
+ fi
+ $PERL -w "$testsrcdir"/extract-testsuite-summary stdout > summary.got \
+ || fatal_ "cannot extract testsuite summary"
+ cat summary.exp
+ cat summary.got
+ if test $use_colors = yes; then
+ # Use cmp, not diff, because the files might contain binary data.
+ compare=cmp
+ else
+ compare=diff
+ fi
+ $compare summary.exp summary.got || Exit 1
+}
+
+br='============================================================================'
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE --add-missing
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check coloring of the testsuite summary.
+
+. ./defs-static || Exit 1
+
+use_colors=yes
+use_vpath=no
+
+. "$testsrcdir"/testsuite-summary-checks.sh || Exit 99
+
+./configure
+
+# ANSI colors.
+red='\e[0;31m'
+grn='\e[0;32m'
+lgn='\e[1;32m'
+blu='\e[1;34m'
+mgn='\e[0;35m'
+brg='\e[1m';
+std='\e[m';
+
+success_header="\
+${grn}${br}${std}
+${grn}Testsuite summary for GNU AutoFoo 7.1${std}
+${grn}${br}${std}"
+
+success_footer=${grn}${br}${std}
+
+failure_header="\
+${red}${br}${std}
+${red}Testsuite summary for GNU AutoFoo 7.1${std}
+${red}${br}${std}"
+
+failure_footer="\
+${red}${br}${std}
+${red}See ./test-suite.log${std}
+${red}Please report to bug-automake@gnu.org${std}
+${red}${br}${std}"
+
+do_check '' <<END
+$success_header
+# TOTAL: 0
+# PASS: 0
+# SKIP: 0
+# XFAIL: 0
+# FAIL: 0
+# XPASS: 0
+# ERROR: 0
+$success_footer
+END
+
+do_check pass.t <<END
+$success_header
+${brg}# TOTAL: 1${std}
+${grn}# PASS: 1${std}
+# SKIP: 0
+# XFAIL: 0
+# FAIL: 0
+# XPASS: 0
+# ERROR: 0
+$success_footer
+END
+
+do_check skip.t <<END
+$success_header
+${brg}# TOTAL: 1${std}
+# PASS: 0
+${blu}# SKIP: 1${std}
+# XFAIL: 0
+# FAIL: 0
+# XPASS: 0
+# ERROR: 0
+$success_footer
+END
+
+do_check xfail.t <<END
+$success_header
+${brg}# TOTAL: 1${std}
+# PASS: 0
+# SKIP: 0
+${lgn}# XFAIL: 1${std}
+# FAIL: 0
+# XPASS: 0
+# ERROR: 0
+$success_footer
+END
+
+do_check fail.t <<END
+$failure_header
+${brg}# TOTAL: 1${std}
+# PASS: 0
+# SKIP: 0
+# XFAIL: 0
+${red}# FAIL: 1${std}
+# XPASS: 0
+# ERROR: 0
+$failure_footer
+END
+
+do_check xpass.t <<END
+$failure_header
+${brg}# TOTAL: 1${std}
+# PASS: 0
+# SKIP: 0
+# XFAIL: 0
+# FAIL: 0
+${red}# XPASS: 1${std}
+# ERROR: 0
+$failure_footer
+END
+
+do_check error.t <<END
+$failure_header
+${brg}# TOTAL: 1${std}
+# PASS: 0
+# SKIP: 0
+# XFAIL: 0
+# FAIL: 0
+# XPASS: 0
+${mgn}# ERROR: 1${std}
+$failure_footer
+END
+
+do_check pass.t xfail.t skip.t <<END
+$success_header
+${brg}# TOTAL: 3${std}
+${grn}# PASS: 1${std}
+${blu}# SKIP: 1${std}
+${lgn}# XFAIL: 1${std}
+# FAIL: 0
+# XPASS: 0
+# ERROR: 0
+$success_footer
+END
+
+do_check pass.t fail.t skip.t <<END
+$failure_header
+${brg}# TOTAL: 3${std}
+${grn}# PASS: 1${std}
+${blu}# SKIP: 1${std}
+# XFAIL: 0
+${red}# FAIL: 1${std}
+# XPASS: 0
+# ERROR: 0
+$failure_footer
+END
+
+do_check pass.t xfail.t xpass.t <<END
+$failure_header
+${brg}# TOTAL: 3${std}
+${grn}# PASS: 1${std}
+# SKIP: 0
+${lgn}# XFAIL: 1${std}
+# FAIL: 0
+${red}# XPASS: 1${std}
+# ERROR: 0
+$failure_footer
+END
+
+do_check skip.t xfail.t error.t <<END
+$failure_header
+${brg}# TOTAL: 3${std}
+# PASS: 0
+${blu}# SKIP: 1${std}
+${lgn}# XFAIL: 1${std}
+# FAIL: 0
+# XPASS: 0
+${mgn}# ERROR: 1${std}
+$failure_footer
+END
+
+do_check pass.t skip.t xfail.t fail.t xpass.t error.t <<END
+$failure_header
+${brg}# TOTAL: 6${std}
+${grn}# PASS: 1${std}
+${blu}# SKIP: 1${std}
+${lgn}# XFAIL: 1${std}
+${red}# FAIL: 1${std}
+${red}# XPASS: 1${std}
+${mgn}# ERROR: 1${std}
+$failure_footer
+END
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check test counts in the testsuite summary, with test drivers allowing
+# multiple test results per test script, and for a huge number of tests.
+# Incidentally, this test also checks that the testsuite summary doesn't
+# give any bug-report address if it's not defined.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+for s in trivial-test-driver extract-testsuite-summary; do
+ cp "$testsrcdir/$s" . || fatal_ "failed to fetch auxiliary script $s"
+done
+
+br='============================================================================'
+
+header="\
+${br}
+Testsuite summary for $me 1.0
+${br}"
+
+footer="\
+${br}
+See ./test-suite.log
+${br}"
+
+echo AC_OUTPUT >> configure.in
+
+cat > Makefile.am << 'END'
+TEST_LOG_DRIVER = $(SHELL) $(srcdir)/trivial-test-driver
+TESTS = all.test
+# Without this, the test driver will be horrendously slow.
+END
+
+cat > all.test <<'END'
+#!/bin/sh
+cat results.txt || { echo ERROR: weird; exit 99; }
+END
+chmod a+x all.test
+
+$PERL -w -e '
+ use warnings FATAL => "all";
+ use strict;
+
+ # FIXME: we would like this to be 1000 or even 10000, but the current
+ # implementation is too slow to handle that :-(
+ my $base = 5;
+ my %count = (
+ TOTAL => $base * 1000,
+ PASS => $base * 700,
+ SKIP => $base * 200,
+ XFAIL => $base * 80,
+ FAIL => $base * 10,
+ XPASS => $base * 7,
+ ERROR => $base * 3,
+ );
+ my @results = qw/PASS SKIP XFAIL FAIL XPASS ERROR/;
+
+ open (RES, ">results.txt") or die "opening results.txt: $!\n";
+ open (CNT, ">count.txt") or die "opening count.txt: $!\n";
+
+ printf CNT "# %-6s %d\n", "TOTAL:", $count{TOTAL};
+ for my $res (@results)
+ {
+ my $uc_res = uc $res;
+ print STDERR "Generating list of $res ...\n";
+ for (1..$count{$res})
+ {
+ print RES "$uc_res: $_\n";
+ }
+ printf CNT "# %-6s %d\n", $res . ":", $count{$res};
+ }
+'
+
+(echo "$header" && cat count.txt && echo "$footer") > summary.exp
+
+$ACLOCAL
+$AUTOMAKE -a
+$AUTOCONF
+
+./configure
+
+($MAKE check || : > make.fail) | tee stdout
+test -f make.fail
+
+$PERL -w extract-testsuite-summary stdout > summary.got
+cat summary.exp
+cat summary.got
+diff summary.exp summary.got || Exit 1
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check test counts in the testsuite summary.
+
+. ./defs-static || Exit 1
+
+use_colors=no
+use_vpath=no
+
+. "$testsrcdir"/testsuite-summary-checks.sh || Exit 99
+
+./configure
+
+header="\
+${br}
+Testsuite summary for GNU AutoFoo 7.1
+${br}"
+
+success_footer=${br}
+
+failure_footer="\
+${br}
+See ./test-suite.log
+Please report to bug-automake@gnu.org
+${br}"
+
+# Corner cases.
+
+do_check '' <<END
+$header
+# TOTAL: 0
+# PASS: 0
+# SKIP: 0
+# XFAIL: 0
+# FAIL: 0
+# XPASS: 0
+# ERROR: 0
+$success_footer
+END
+
+do_check pass.t <<END
+$header
+# TOTAL: 1
+# PASS: 1
+# SKIP: 0
+# XFAIL: 0
+# FAIL: 0
+# XPASS: 0
+# ERROR: 0
+$success_footer
+END
+
+do_check fail.t <<END
+$header
+# TOTAL: 1
+# PASS: 0
+# SKIP: 0
+# XFAIL: 0
+# FAIL: 1
+# XPASS: 0
+# ERROR: 0
+$failure_footer
+END
+
+# Some simpler checks, with low or moderate number of tests.
+
+do_check skip.t skip2.t skip3.t xfail.t xfail2.t <<END
+$header
+# TOTAL: 5
+# PASS: 0
+# SKIP: 3
+# XFAIL: 2
+# FAIL: 0
+# XPASS: 0
+# ERROR: 0
+$success_footer
+END
+
+do_check pass.t pass2.t xfail.t xpass.t error.t error2.t <<END
+$header
+# TOTAL: 6
+# PASS: 2
+# SKIP: 0
+# XFAIL: 1
+# FAIL: 0
+# XPASS: 1
+# ERROR: 2
+$failure_footer
+END
+
+pass_count=22
+skip_count=19
+xfail_count=21
+fail_count=18
+xpass_count=23
+error_count=17
+tests_count=120
+
+pass=` seq_ 1 $pass_count | sed 's/.*/pass-&.t/'`
+skip=` seq_ 1 $skip_count | sed 's/.*/skip-&.t/'`
+xfail=`seq_ 1 $xfail_count | sed 's/.*/xfail-&.t/'`
+fail=` seq_ 1 $fail_count | sed 's/.*/fail-&.t/'`
+xpass=`seq_ 1 $xpass_count | sed 's/.*/xpass-&.t/'`
+error=`seq_ 1 $error_count | sed 's/.*/error-&.t/'`
+
+do_check $pass $skip $xfail $fail $xpass $error <<END
+$header
+# TOTAL: $tests_count
+# PASS: $pass_count
+# SKIP: $skip_count
+# XFAIL: $xfail_count
+# FAIL: $fail_count
+# XPASS: $xpass_count
+# ERROR: $error_count
+$failure_footer
+END
+
+# Mild stress test with a lot of test scripts.
+
+tests_count=1888
+pass_count=1403
+skip_count=292
+xfail_count=41
+fail_count=126
+xpass_count=17
+error_count=9
+
+pass=` seq_ 1 $pass_count | sed 's/.*/pass-&.t/'`
+skip=` seq_ 1 $skip_count | sed 's/.*/skip-&.t/'`
+xfail=`seq_ 1 $xfail_count | sed 's/.*/xfail-&.t/'`
+fail=` seq_ 1 $fail_count | sed 's/.*/fail-&.t/'`
+xpass=`seq_ 1 $xpass_count | sed 's/.*/xpass-&.t/'`
+error=`seq_ 1 $error_count | sed 's/.*/error-&.t/'`
+
+do_check $pass $skip $xfail $fail $xpass $error <<END
+$header
+# TOTAL: $tests_count
+# PASS: $pass_count
+# SKIP: $skip_count
+# XFAIL: $xfail_count
+# FAIL: $fail_count
+# XPASS: $xpass_count
+# ERROR: $error_count
+$failure_footer
+END
+
+:
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Check that the global testsuite log file referenced in the testsuite
+# summary and in the global testsuite log itself is correct.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+mv configure.in configure.stub
+
+cat > fail << 'END'
+#!/bin/sh
+exit 1
+END
+chmod a+x fail
+
+cat configure.stub - > configure.in <<'END'
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+TEST_SUITE_LOG = my_test_suite.log
+TESTS = fail
+END
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+mkdir build
+cd build
+
+../configure
+
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+grep '^See \./my_test_suite\.log$' stdout
+
+mkdir bar
+TEST_SUITE_LOG=bar/bar.log $MAKE -e check >stdout && { cat stdout; Exit 1; }
+cat stdout
+grep '^See \./bar/bar\.log$' stdout
+
+cd ..
+
+echo SUBDIRS = sub > Makefile.am
+mkdir sub
+echo TESTS = fail > sub/Makefile.am
+mv fail sub
+
+cat configure.stub - > configure.in <<'END'
+AC_CONFIG_FILES([sub/Makefile])
+AC_OUTPUT
+END
+
+$ACLOCAL --force
+$AUTOCONF --force
+$AUTOMAKE
+
+./configure
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+grep '^See sub/test-suite\.log$' stdout
+cd sub
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+grep '^See sub/test-suite\.log$' stdout
+cd ..
+
+TEST_SUITE_LOG=foo.log $MAKE -e check >stdout && { cat stdout; Exit 1; }
+cat stdout
+grep '^See sub/foo\.log$' stdout
+
+:
# Make sure that --program-transform works.
-required=gcc
+required=cc
. ./defs || Exit 1
cat >>configure.in <<'END'
bin_SCRIPTS = h.sh
man_MANS = h.1
+.PHONY: test-install
test-install: install
test -f inst/bin/gnu-h$(EXEEXT)
test -f inst/bin/gnu-h.sh
EOF
cat >h.c <<'EOF'
-int
-main ()
+int main (void)
{
return 0;
}
$MAKE installdirs
test -d inst/bin
test -d inst/man/man1
+
+:
# Make sure that --program-transform works even when multiple files are
# collapsed.
+required=cc
. ./defs || Exit 1
cat >>configure.in <<'END'
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+#
+# Test driver for a very simple test protocol used by the Automake
+# testsuite to check support for custom test drivers allowing for more
+# test results per test script.
+#
+# The exit status of the wrapped script is ignored. Lines in its stdout
+# and stderr beginning with `PASS', `FAIL', `XFAIL', `XPASS', `SKIP' or
+# `ERROR' count as a test case result with the obviously-corresponding
+# outcome. Every other line is ignored for what concerns the testsuite
+# outcome.
+#
+# This script is used at least by the `driver-custom-multitest*.test'
+# tests.
+#
+
+# Help to avoid typo-related bugs.
+set -u
+
+## Option parsing.
+
+test_name=INVALID.NAME
+log_file=BAD.LOG
+trs_file=BAD.TRS
+while test $# -gt 0; do
+ case $1 in
+ --test-name) test_name=$2; shift;;
+ --log-file) log_file=$2; shift;;
+ --trs-file) trs_file=$2; shift;;
+ # Ignored.
+ --expect-failure) shift;;
+ --color-tests) shift;;
+ --enable-hard-errors) shift;;
+ # Explicitly terminate option list.
+ --) shift; break;;
+ # Shouldn't happen
+ *) echo "$0: invalid option/argument: '$1'" >&2; exit 2;;
+ esac
+ shift
+done
+
+## Log file header.
+{
+ echo "RUN: $test_name"
+ echo "RUN: $test_name" | sed 's/./=/g'
+ echo
+} > $log_file
+
+## Run the test script, get test cases results, display them on console.
+
+"$@" 2>&1 | tee -a $log_file | (
+ i=0 st=0
+ exec 5> $trs_file
+ while read line; do
+ result=
+ case $line in
+ PASS:*) result=PASS ;;
+ FAIL:*) result=FAIL ;;
+ XPASS:*) result=XPASS ;;
+ XFAIL:*) result=XFAIL ;;
+ SKIP:*) result=SKIP ;;
+ ERROR:*) result=ERROR ;;
+ esac
+ if test -n "$result"; then
+ case $result in FAIL|XPASS|ERROR) st=1;; esac
+ # Output testcase result to console.
+ echo "$result: $test_name"
+ # Register testcase outcome for the log file.
+ echo ":test-result: $line" >&5
+ echo >&5
+ fi
+ done
+ if test $st -eq 0; then
+ recheck=no
+ copy_in_global_log=no
+ else
+ recheck=yes
+ copy_in_global_log=yes
+ fi
+ echo ":recheck: $recheck" >&5
+ echo ":copy-in-global-log: $copy_in_global_log" >&5
+ exec 5>&-
+) | awk '{ print $0 ", testcase " NR }'
+
+## And we're done.
+
+exit 0
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Make sure dvi target recurses into subdir.
-# Reported by Pavel Roskin
+# Reported by Pavel Roskin.
. ./defs || Exit 1
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Check for subdir Texinfo (PR/343)
+# Check for subdir Texinfo (PR/343).
# Also make sure we do not distribute too much foo.info* files (Report
-# from Vincent Lefevre)
+# from Vincent Lefevre).
required='makeinfo tex texi2dvi-o'
. ./defs || Exit 1
./configure
$MAKE dvi
-ls -l # for debugging
-test -f main.sa # sanity check
+ls -l # For debugging.
+test -f main.sa # Sanity check.
$MAKE clean
-ls -l # for debugging
+ls -l # For debugging.
test x"`echo main.*`" = x"main.texi"
TAR_OPTIONS= $MAKE distcheck
: > main-1
: > sub/main-1
-# Break main.texi
+# Break main.texi.
$sleep
cp main.texi main.old
cat > main.texi << 'END'
test -f main
test -f main-1
-# Restore main.texi, and break sub/main.texi
+# Restore main.texi, and break sub/main.texi.
cp main.texi sub/main.texi
mv main.old main.texi
$MAKE && Exit 1
test ! -d rec/main3.html
$MAKE install-html
-test -f share/txinfo21/html/main.html
-test -f share/txinfo21/html/main2.html
-test -d share/txinfo21/html/main3.html
+test -f share/$me/html/main.html
+test -f share/$me/html/main2.html
+test -d share/$me/html/main3.html
$MAKE uninstall
-test ! -f share/txinfo21/html/main.html
-test ! -f share/txinfo21/html/main2.html
-test ! -d share/txinfo21/html/main3.html
+test ! -f share/$me/html/main.html
+test ! -f share/$me/html/main2.html
+test ! -d share/$me/html/main3.html
$MAKE install-dvi
-test -f share/txinfo21/dvi/main.dvi
-test -f share/txinfo21/dvi/main2.dvi
-test -f share/txinfo21/dvi/main3.dvi
+test -f share/$me/dvi/main.dvi
+test -f share/$me/dvi/main2.dvi
+test -f share/$me/dvi/main3.dvi
$MAKE uninstall
-test ! -f share/txinfo21/dvi/main.dvi
-test ! -f share/txinfo21/dvi/main2.dvi
-test ! -f share/txinfo21/dvi/main3.dvi
+test ! -f share/$me/dvi/main.dvi
+test ! -f share/$me/dvi/main2.dvi
+test ! -f share/$me/dvi/main3.dvi
-(dvips --help 2>/dev/null >/dev/null) || Exit 77
+dvips --help || skip_ "dvips is missing"
$MAKE install-ps
-test -f share/txinfo21/ps/main.ps
-test -f share/txinfo21/ps/main2.ps
-test -f share/txinfo21/ps/main3.ps
+test -f share/$me/ps/main.ps
+test -f share/$me/ps/main2.ps
+test -f share/$me/ps/main3.ps
$MAKE uninstall
-test ! -f share/txinfo21/ps/main.ps
-test ! -f share/txinfo21/ps/main2.ps
-test ! -f share/txinfo21/ps/main3.ps
+test ! -f share/$me/ps/main.ps
+test ! -f share/$me/ps/main2.ps
+test ! -f share/$me/ps/main3.ps
-(pdfetex --help 2>/dev/null >/dev/null) ||
- (pdftex --help 2>/dev/null >/dev/null) || Exit 77
+pdfetex --help || pdftex --help \
+ || skip_ "pdftex and pdfetex are both missing"
$MAKE install-pdf
-test -f share/txinfo21/pdf/main.pdf
-test -f share/txinfo21/pdf/main2.pdf
-test -f share/txinfo21/pdf/main3.pdf
-test -f share/txinfo21/pdf/hello
+test -f share/$me/pdf/main.pdf
+test -f share/$me/pdf/main2.pdf
+test -f share/$me/pdf/main3.pdf
+test -f share/$me/pdf/hello
$MAKE uninstall
-test ! -f share/txinfo21/pdf/main.pdf
-test ! -f share/txinfo21/pdf/main2.pdf
-test ! -f share/txinfo21/pdf/main3.pdf
-test ! -f share/txinfo21/pdf/hello
+test ! -f share/$me/pdf/main.pdf
+test ! -f share/$me/pdf/main2.pdf
+test ! -f share/$me/pdf/main3.pdf
+test ! -f share/$me/pdf/hello
+
+:
# Check that info files are built in builddir when needed.
# Test with subdir Texinfo.
-# (Similar to txinfo13.test, plus DISTCLEANFILES.)
-# (See also txinfo24.test and txinfo25.test)
+# (Similar to txinfo13.test, plus DISTCLEANFILES).
+# (See also txinfo24.test and txinfo25.test).
required='makeinfo tex texi2dvi-o'
. ./defs || Exit 1
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Check that info files are built in builddir when needed.
-# (Similar to txinfo16.test, plus CLEANFILES.)
-# (See also txinfo23.test and txinfo25.test)
+# (Similar to txinfo16.test, plus CLEANFILES).
+# (See also txinfo23.test and txinfo25.test).
required='makeinfo tex texi2dvi-o'
. ./defs || Exit 1
test -f main.info
# Make sure stamp-vti is older that version.texi.
-# (A common situation in a real tree.)
+# (A common situation in a real tree).
test -f stamp-vti
test -f version.texi
$sleep
cd build
../configure
$MAKE
-# main.info should be rebuilt in the current directory
+# main.info should be rebuilt in the current directory.
test -f main.info
test ! -f ../main.info
$MAKE dvi
# Check that info files are built in builddir and in srcdir can safely
# co-exist. This setup is obtained by having two info files, only one
# of which being cleaned.
-# (Similar to txinfo16.test, plus CLEANFILES.)
-# (See also txinfo23.test and txinfo24.test)
+# (Similar to txinfo16.test, plus CLEANFILES).
+# (See also txinfo23.test and txinfo24.test).
required='makeinfo tex texi2dvi-o'
. ./defs || Exit 1
test -f other.info
# Make sure stamp-vti is older that version.texi.
-# (A common situation in a real tree.)
+# (A common situation in a real tree).
# This is needed to test the "subtle" issue described below.
test -f stamp-vti
test -f version.texi
$MAKE
$MAKE distclean
-# We do not require this to work with a directory that contains white space.
case `pwd` in
- *\ * | *\ *) Exit 77;;
+ *\ * | *\ *)
+ skip_ "this test might fail in a directory containing white spaces";;
esac
mkdir build
test ! -f _inst/info/main.info
test -f ../main.info
-# multiple uninstall should not fail.
+# Multiple uninstall should not fail.
$MAKE uninstall
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Check that info files are built in builddir when needed.
-# Similar to txinfo24.test, but obfuscating filenames with variable references.
+# Similar to txinfo24.test, but obfuscating filenames with variable
+# references.
# Report from Ralf Corsepius.
required='makeinfo tex texi2dvi-o'
test -f main.info
# Make sure stamp-vti is older that version.texi.
-# (A common situation in a real tree.)
+# (A common situation in a real tree).
test -f stamp-vti
test -f version.texi
$sleep
cd build
../configure
$MAKE
-# main.info should be rebuilt in the current directory
+# main.info should be rebuilt in the current directory.
test -f main.info
test ! -f ../main.info
$MAKE dvi
$AUTOMAKE -Wno-override
-# There is only one definition of INFO_DEPS
+# There is only one definition of INFO_DEPS ...
test 1 = `grep '^INFO_DEPS.*=' Makefile.in | wc -l`
-# and it is the right one.
+# ... and it is the right one.
grep '^INFO_DEPS *= *foo.info *$' Makefile.in
# Likewise with AC_SUBST.
$AUTOMAKE -Wno-override
-# There is only one definition of INFO_DEPS
+# There is only one definition of INFO_DEPS ...
test 1 = `grep '^INFO_DEPS.*=' Makefile.in | wc -l`
-# and it is the right one.
+# ... and it is the right one.
grep '^INFO_DEPS *= *@INFO_DEPS@ *$' Makefile.in
:
# we only want them to appear once. But grepping them all would be
# overkill.
for t in info dist-info dvi-am install-html uninstall-pdf-am; do
- $EGREP "(^| )$t*.:" Makefile.in # help in debugging
+ $EGREP "(^| )$t*.:" Makefile.in # For debugging.
test `$EGREP -c "(^| )$t(:| *.:)" Makefile.in` -eq 1
done
# A simple Hello World for UPC.
. ./defs || Exit 1
+
cat >> configure.in << 'END'
AM_PROG_UPC
AC_OUTPUT
$AUTOMAKE
$AUTOCONF
-./configure
+./configure || Exit $?
$MAKE distcheck
+
+:
# Test that C and Unified Parallel C link safely.
. ./defs || Exit 1
+
cat >> configure.in << 'END'
AC_PROG_CC
AM_PROG_UPC
$AUTOMAKE
$AUTOCONF
-./configure
+./configure || Exit $?
$MAKE distcheck
+
+:
$AUTOCONF
$AUTOMAKE -a
-./configure || Exit 77
+./configure || skip_ "configure failure"
$MAKE
-# test rebuild rules
+# Test rebuild rules.
rm -f src/zardoz.h
$MAKE -C src zardoz.h
$MAKE
$MAKE distcheck
-# test rebuild rules from builddir
+# Test rebuild rules from builddir.
touch ../src/zardoz.vala
$MAKE
$AUTOCONF
$AUTOMAKE -a
-./configure || Exit 77
+./configure || skip_ "configure failure"
$MAKE
$MAKE distcheck
$MAKE distclean
$AUTOCONF
$AUTOMAKE -a
-./configure || Exit 77
+./configure || skip_ "configure failure"
$MAKE
$MAKE distcheck
$MAKE distclean
$ACLOCAL
$AUTOMAKE -a
$AUTOCONF
-./configure "VALAC=$cwd/valac"
+./configure "VALAC=$cwd/valac" || Exit $?
sed 's/AM_PROG_VALAC.*/AM_PROG_VALAC([9999.9])/' < configure.in >t
mv -f t configure.in
sed 's/AM_PROG_VALAC.*/AM_PROG_VALAC([1.2.3])/' < configure.in >t
mv -f t configure.in
$AUTOCONF --force
-./configure "VALAC=$cwd/valac"
+./configure "VALAC=$cwd/valac" || Exit $?
+
:
$AUTOCONF
$AUTOMAKE -a
-./configure || Exit 77
+./configure || skip_ "configuration step failure"
$MAKE
+:
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Test various variable definitions that include an '=' sign
-# From Raja R Harinath
+# Test various variable definitions that include an '=' sign.
+# From Raja R Harinath.
. ./defs || Exit 1
# Lines number are printed in error message.
# Use them to make sure errors are diagnosed against the right lines.
-# No error expected for these lines.
-grep 1: stderr
-grep 2: stderr && Exit 1
-grep 3: stderr && Exit 1
-grep 4: stderr
-grep 5: stderr
-grep 6: stderr && Exit 1
-grep 7: stderr
-grep 8: stderr
-grep 9: stderr
-grep 10: stderr && Exit 1
-grep 11: stderr && Exit 1
-grep 12: stderr && Exit 1
+# No error expected apart from those on these lines.
+grep -v '^Makefile\.am:[145789]:' stderr | grep . && Exit 1
# Now check some individual values.
-grep 'shell echo' stderr
+grep ':1:.*shell echo' stderr
grep 'nextvariableisbad' stderr && Exit 1
-grep 'addsuffix' stderr
-grep 'bad boy' stderr
-grep 'ok' stderr && Exit 1
-grep 'three errors' stderr
-grep 'on this' stderr
-grep 'long line' stderr
-grep 'o u c h' stderr
-grep 'wildcard' stderr
-grep 'another Error' stderr
-grep 'thisis' stderr && Exit 1
-grep 'here' stderr && Exit 1
+grep ':4:.*addsuffix' stderr
+grep ':5:.*bad boy' stderr
+grep ':7:.*three errors' stderr
+grep ':7:.*on this' stderr
+grep ':7:.*long line' stderr
+grep ':8:.*o u c h' stderr
+grep ':8:.*wildcard' stderr
+grep ':9:.*another Error' stderr
-# None of these errors be diagnosed with -Wno-portability
+$EGREP 'ok|thisis|here' stderr && Exit 1
+
+# None of these errors be diagnosed with `-Wno-portability'.
$AUTOMAKE -Wno-portability
# Likewise if we add this in the Makefile.am
amver=`$AUTOMAKE --version | sed -e 's/.* //;1q'`
+# Does the extracted version number seems legit?
+case $amver in
+ *[0-9].[0-9]*) ;;
+ *) fatal_ "couldn't extract version number from automake" ;;
+esac
+
cat > Makefile.am << END
AUTOMAKE_OPTIONS = $amver
END
$ACLOCAL
$AUTOMAKE
+
+:
# is to make sure Makefile.ins get rebuilt when a m4_included file
# changes -- we don't support this feature on non-GNU Makes).
-required='makeinfo tex texi2dvi GNUmake'
+required='makeinfo tex texi2dvi'
. ./defs || Exit 1
cat >configure.in <<END
$sleep
echo 'm4_define([THE_VERSION], [3.141])' > version.m4
+using_gmake || $MAKE Makefile
$MAKE distcheck
./configure --version | grep '3\.141'
grep '3\.141' version.texi
$ACLOCAL
$AUTOMAKE
+grep VPATH Makefile.in # For debugging.
+grep '^VPATH = zardoz$' Makefile.in
grep 'VPATH.*@srcdir@' Makefile.in && Exit 1
-Exit 0
+
+:
# to redefine the name of the distdir as well.
distdir=$me-7.45.3a
-# This should work without tex, texinfo or makeinfo
+# This should work without tex, texinfo or makeinfo.
TEX=false TEXI2DVI=false MAKEINFO=false
export TEX TEXI2DVI MAKEINFO
# Required when using Texinfo.
: > texinfo.tex
-cp "$testsrcdir/../lib/mdate-sh" .
+cp "$top_testsrcdir"/lib/mdate-sh .
$ACLOCAL
$AUTOCONF
&& day=`LC_ALL=C date '+%d'` && test -n "$day" \
&& month=`LC_ALL=C date '+%B'` && test -n "$month" \
&& year=`LC_ALL=C date '+%Y'`&& test -n "$year" \
- || { echo "$me: 'date' is not POSIX-compliant enough"; Exit 77; }
+ || skip_ "'date' is not POSIX-compliant enough"
day=`echo "$day" | sed 's/^0//'`
-# This test requires a grep program that can work on non-text input.
-(echo '\ex\a' | grep x) || {
- echo "$me: grep doesn't work on input that is not pure text" >&2
- Exit 77
-}
+(echo '\ex\a' | grep x) \
+ || skip_ "grep doesn't work on input that is not pure text"
cat > configure.in << END
AC_INIT([$me], [123.456])
./configure
$MAKE all dvi
-# debugging & sanity checks
+# Debugging & sanity checks.
ls -l
cat version.texi
cat foo.info
ok ()
{
- AUTOMAKE_run 0 $*
+ AUTOMAKE_run $*
test ! -s stderr
}
ko ()
{
- AUTOMAKE_run 0 $*
+ AUTOMAKE_run $*
grep '^Makefile\.am:.*:=.*not portable' stderr
test `wc -l <stderr` -eq 1
}
# created below; thus causing traces for the old configure.in to be
# used. We could do `$sleep', but it's faster to erase the
# directory. (Erase autom4te*.cache, not autom4te.cache, because some
-# bogus installations of Autoconf use a versioned cache.)
+# bogus installations of Autoconf use a versioned cache).
rm -rf autom4te*.cache
# If we add a global -Wnone, all warnings should disappear.
# succeed if make is run a second time immediately afterwards.
# Report from Harlan Stenn.
-required=GNUmake
. ./defs || Exit 1
echo AC_OUTPUT>>configure.in
foo_SOURCES = unused
END
+# This repetition is deliberate; see heading comments.
$MAKE && Exit 1
-
$MAKE && Exit 1
:
# The issue would not manifest with threaded execution.
unset AUTOMAKE_JOBS || :
-AUTOMAKE_run 0 -Wno-error
+AUTOMAKE_run -Wno-error
grep 'VAR multiply defined' stderr
AUTOMAKE_JOBS=2
export AUTOMAKE_JOBS
-AUTOMAKE_run 0 -Wno-error
+AUTOMAKE_run -Wno-error
grep 'VAR multiply defined' stderr
:
grep 'SUB multiply defined' stderr
test `grep -c 'warnings are treated as errors' stderr` -eq 1
-AUTOMAKE_run 0 -Wno-error
+AUTOMAKE_run -Wno-error
grep 'VAR multiply defined' stderr
grep 'SUB multiply defined' stderr
grep 'warnings are treated as errors' stderr && Exit 1
--- /dev/null
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Wrap en existing test script of the automake testsuite by pre-defining
+# some influential variables and then sourcing it.
+
+set -e
+
+# Will be properly overridden once the wrapped test script is sourced.
+fatal_ () { echo "$0: $*" >&2; exit 99; }
+
+# Ensure proper definition of $testsrcdir.
+. ./defs-static || fatal_ "couldn't source ./defs-static"
+test x"$testsrcdir" != x || fatal_ "\$testsrcdir is empty or undefined"
+
+echo "args: $*" # For debugging.
+
+typ=
+while test $# -gt 0; do
+ case $1 in
+ --define)
+ test $# -ge 3 || fatal_ "option \`$1': two arguments required"
+ echo "define: $2='$3'" # For debugging.
+ eval "$2=\$3"
+ shift; shift;;
+ --type)
+ test $# -ge 2 || fatal_ "option \`$1': argument required"
+ typ=$2
+ shift;;
+ --)
+ shift; break;;
+ *)
+ break;;
+ -*)
+ fatal_ "invalid option: \`$1'";;
+ esac
+ shift
+done
+
+case $typ in
+ "") fatal_ "suffix not specified";;
+ *.*) fatal_ "invalid suffix \`$typ' (contains dot)";;
+esac
+
+case $# in
+ 0) fatal_ "missing argument";;
+ 1) ;;
+ *) fatal_ "too many arguments";;
+esac
+
+# We must let the code in ./defs which kind of test script it is
+# dealing with -- TAP or "plain". It won't be able to guess
+# automatically, since it uses `$0' for such a guess, and with
+# the present usage `$0' is always `wrap-tests.sh'.
+case $1 in
+ *-w.test_$typ) using_tap=no suf=test;;
+ *-w.tap_$typ) using_tap=yes suf=tap;;
+ *) fatal_ "invalid argument \`$1'";;
+esac
+
+test_name=`expr x/"$1" : ".*/\\\\(.*\\\\)-w\\\\.[^.][^.]*$"` \
+ && test -n "$test_name" \
+ || fatal_ "couldn't extract test name"
+shift
+
+# This is required to have the wrapped test use a proper temporary
+# directory to run into.
+me=${test_name}-w
+# In the spirit of VPATH, we prefer a test in the build tree
+# over one in the source tree.
+for dir in . "$testsrcdir"; do
+ . "$dir/$test_name.$suf"
+ exit $?
+done
+
+fatal_ "cannot find wrapped test \`$test_name'"
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Check support for no-dist-gzip with xz
+# Check support for no-dist-gzip with xz.
required=xz
. ./defs || Exit 1
# Basic semantic checks on Yacc support.
-required=yacc
+required='cc yacc'
. ./defs || Exit 1
cat >> configure.in << 'END'
# like an "intermediate file" in the GNU make sense).
test -f parse.c
-echo a | ./foo
-echo b | ./foo && Exit 1
+if cross_compiling; then :; else
+ echo a | ./foo
+ echo b | ./foo && Exit 1
+fi
# The generated file `parse.c' must be shipped.
$MAKE echo-distcom
test -f parse.c
$MAKE distclean
test -f parse.c
-./configure # we must re-create `Makefile'
+./configure # Re-create `Makefile'.
$MAKE maintainer-clean
test ! -f parse.c
# are cleaned by "make clean", while .c and .h files derived from
# distributed .y sources are cleaned by "make maintainer-clean".
-required=yacc
+required='cc yacc'
. ./defs || Exit 1
cat >> configure.in << 'END'
done
cp config.sav config.status
-./config.status # re-create Makefile
+./config.status # Re-create `Makefile'.
$MAKE maintainer-clean
ls -l . sub1 sub2
test ! -r sub2/bar-parse.h
cp config.sav config.status
-./config.status # re-create Makefile
+./config.status # Re-create `Makefile'.
# The distribution must work correctly, assuming the user has
# the proper tools to process yacc files.
# Tests on basic Yacc support for when we have -d in YFLAGS, AM_YFLAGS
# or maude_YFLAGS.
-required=yacc
+required='cc yacc'
. ./defs || Exit 1
tab=' '
# Please keep this in sync with sister test `yaccvpath.test'.
-required=yacc
+required='cc yacc'
. ./defs || Exit 1
cat >> configure.in << 'END'
AM_YFLAGS = -d
END
-# Original parser, with `foobar'
+# Original parser, with `foobar'.
cat > parse.y << 'END'
%{
int yylex () {return 0;}
$sleep
-# New parser, with `fubar'
+# New parser, with `fubar'.
cat > ../parse.y << 'END'
%{
int yylex () {return 0;}
$sleep
-# New parser, with `maude'
+# New parser, with `maude'.
cat > ../parse.y << 'END'
%{
int yylex () {return 0;}
./configure
$MAKE distdir
-# Yacc-derived C source and header files must be built and distributed
+# Yacc-derived C source and header files must be built and distributed.
test -f sub/parse.c
test -f sub/parse.h
# Check that distributed Yacc-generated parsers are not uselessly
# remade from an unpacked distributed tarball.
-required=yacc
+required='cc yacc'
. ./defs || Exit 1
cat >> configure.in << 'END'
# Checks for .c and .h files derived from non-distributed .y sources.
-required=yacc
+required='cc yacc'
. ./defs || Exit 1
cat >> configure.in << 'END'
cp Makefile.am Makefile.src
$AUTOMAKE -a
-# If zardoz.h IS mentioned, fail
+# If zardoz.h IS mentioned, fail.
$FGREP 'zardoz.h' Makefile.in && Exit 1
cp Makefile.src Makefile.am
echo 'AM_YFLAGS = -d' >> Makefile.am
$AUTOMAKE
-# If zardoz.h is NOT mentioned, fail
+# If zardoz.h is NOT mentioned, fail.
$FGREP 'zardoz.h' Makefile.in
cp Makefile.src Makefile.am
echo 'AM_YFLAGS = ' >> Makefile.am
$AUTOMAKE
-# If zardoz.h IS mentioned, fail
+# If zardoz.h IS mentioned, fail.
$FGREP 'zardoz.h' Makefile.in && Exit 1
cp Makefile.src Makefile.am
grep 'YFLAGS.* user variable' stderr
grep 'AM_YFLAGS.* instead' stderr
$AUTOMAKE -Wno-gnu
-# If zardoz.h is NOT mentioned, fail
+# If zardoz.h is NOT mentioned, fail.
$FGREP 'zardoz.h' Makefile.in
cp Makefile.src Makefile.am
echo 'YFLAGS = ' >> Makefile.am
$AUTOMAKE -Wno-gnu
-# If zardoz.h IS mentioned, fail
+# If zardoz.h IS mentioned, fail.
$FGREP 'zardoz.h' Makefile.in && Exit 1
:
# Some simple tests of ylwrap functionality.
-required='yacc gcc'
+required='cc yacc'
. ./defs || Exit 1
cat >> configure.in << 'END'
# Second parser.
cat > bar.y << 'END'
%{
-int yylex () {return 0;}
+int yylex () { return 0; }
void yyerror (char *s) {}
%}
%%
END
cat > foo.c << 'END'
-int main () { return 0; }
+int main (void)
+{
+ return 0;
+}
END
$ACLOCAL
cat > configure.in << 'END'
AC_INIT([yacc6], [1.0])
-# `aux' is not an acceptable file/directory name on Windows systems
+# `aux' is not an acceptable file/directory name on Windows systems.
AC_CONFIG_AUX_DIR([aux1])
AM_INIT_AUTOMAKE
AC_CONFIG_FILES([Makefile sub/Makefile])
# Also check that the sources of the generated parser are distributed.
# PR/47.
-required=yacc
+required='cc yacc'
. ./defs || Exit 1
cat >> configure.in << 'END'
test -f foo.h
test -f foo.c
# ... but maintainer-clean should.
-./configure # we must re-create `Makefile'
+./configure # Re-create `Makefile'.
$MAKE maintainer-clean
test ! -f foo.h
test ! -f foo.c
# Test for subdir parsers.
-required="gcc yacc"
+required='cc yacc'
. ./defs || Exit 1
$MAKE test1
# Aside of the rest of this test, let's see if we can recover from
-# parse.h removal
+# parse.h removal.
test -f foo/parse.h
rm -f foo/parse.h
$MAKE foo/parse.h
test -f ./ylwrap
cd sub
-# Regenerate Makefile (automatic in GNU Make, but not in other Makes)
+# Regenerate Makefile (automatic in GNU Make, but not in other Makes).
./config.status
$MAKE test2
# Removal recovery rules for headers should not remove files with `make -n'.
-required=yacc
+required='cc yacc'
. ./defs || Exit 1
cat >> configure.in << 'END'
# Please keep this in sync with sister test `yacc-d-vpath.test'.
-required=yacc
+required='cc yacc'
. ./defs || Exit 1
distdir=$me-1.0
foo_SOURCES = parse.y foo.c
END
-# Original parser, with `foobar'
+# Original parser, with `foobar'.
cat > parse.y << 'END'
%{
int yylex () {return 0;}
$sleep
-# New parser, with `fubar'
+# New parser, with `fubar'.
cat > ../parse.y << 'END'
%{
int yylex () {return 0;}
$sleep
-# New parser, with `maude'
+# New parser, with `maude'.
cat > ../parse.y << 'END'
%{
int yylex () {return 0;}
# Check that automake can cope with user-redefinition of $(YFLAGS)
# at configure time and/or at make time.
-required=yacc
+required='cc yacc'
. ./defs || Exit 1
unset YFLAGS || :
cat > Makefile1.am <<'END'
bin_PROGRAMS = foo
foo_SOURCES = foo.y
-## dummy comment to keep line count right
+## This is a dummy comment to keep line count right.
if COND
YFLAGS = foo
endif COND
: > ylwrap
-LC_ALL=C; export LC_ALL # For grep regexes below.
+LC_ALL=C; export LC_ALL; # For grep regexes below.
AUTOMAKE_fails -Wnone -Wunsupported Makefile
grep '^Makefile\.am:5:.*AM_YFLAGS.* conditional contents' stderr
. ./defs || Exit 1
cat >> configure.in <<'END'
-AC_PROG_CC
+AC_SUBST([CC], [false])
AC_PROG_YACC
AM_CONDITIONAL([COND], [test x"$cond" = x"yes"])
AC_OUTPUT
YACC=fake-yacc; export YACC
cat > Makefile.am <<'END'
+AUTOMAKE_OPTIONS = no-dependencies
bin_PROGRAMS = foo bar
foo_SOURCES = foo.y main.c
bar_SOURCES = $(foo_SOURCES)
endif !COND
END
-cat > main.c <<'END'
-int main (void)
-{
- return 0;
-}
-END
-
: > foo.y
$ACLOCAL
$EGREP '(YFLAGS|yflags|am__append)' Makefile.in # For debugging.
./configure cond=yes
-$MAKE
+$MAKE foo.c bar-foo.c
-ls -l
cat foo.c
cat bar-foo.c
ls -l
./configure cond=no
-$MAKE
+$MAKE foo.c bar-foo.c
-ls -l
cat foo.c
cat bar-foo.c
# in Makefile.am (even if that is extremely bad practice, because that
# variable is user-reserved).
-required=yacc
+required='cc yacc'
. ./defs || Exit 1
cat >> configure.in <<'END'
$ACLOCAL
$AUTOMAKE -a -Wno-gnu
-$EGREP '(foo|YFLAGS)' Makefile.in # for debugging
+$EGREP '(foo|YFLAGS)' Makefile.in # For debugging.
grep '^foo.h *:' Makefile.in
$AUTOCONF
$AUTOMAKE -a
-$EGREP '(foo|bar|YFLAGS)' Makefile.in # for debugging
+$EGREP '(foo|bar|YFLAGS)' Makefile.in # For debugging.
grep '^foo.h *:' Makefile.in
grep '^bar-bar.h *:' Makefile.in
$AUTOMAKE
-$EGREP 'parser|YFLAGS' Makefile.in # for debugging
+$EGREP 'parser|YFLAGS' Makefile.in # For debugging.
grep '^parser.h *:' Makefile.in
:
# Check that $(YFLAGS) takes precedence over both $(AM_YFLAGS) and
# $(foo_YFLAGS).
# Please keep this in sync with the sister tests yflags2.test, lflags.test
-# and lflags2.test
+# and lflags2.test.
. ./defs || Exit 1
unset YACC || :
cat >> configure.in <<'END'
-AC_PROG_CC
+AC_SUBST([CC], [false])
# Simulate presence of Yacc using our fake-yacc script.
AC_SUBST([YACC], ['$(abs_top_srcdir)'/fake-yacc])
AC_OUTPUT
END
cat > Makefile.am <<'END'
+AUTOMAKE_OPTIONS = no-dependencies
bin_PROGRAMS = foo bar
foo_SOURCES = main.c foo.y
bar_SOURCES = main.c bar.y
grep '\$(YFLAGS).*\$(bar_YFLAGS)' Makefile.in && Exit 1
grep '\$(YFLAGS).*\$(AM_YFLAGS)' Makefile.in && Exit 1
-echo 'int main(void){ return 0; }' > main.c
: > foo.y
: > bar.y
$AUTOCONF
./configure
-env YFLAGS=__user_flags__ $MAKE -e
-
-ls -l
+env YFLAGS=__user_flags__ $MAKE -e foo.c bar-bar.c
cat foo.c
-grep '__am_flags__.*__user_flags__' foo.c
cat bar-bar.c
+
+grep '__am_flags__.*__user_flags__' foo.c
grep '__bar_flags__.*__user_flags__' bar-bar.c
:
# Check that $(YFLAGS) takes precedence over both $(AM_YFLAGS) and
# $(foo_YFLAGS).
# Please keep this in sync with the sister tests yflags.test, lflags.test
-# and lflags2.test
+# and lflags2.test.
-required=g++ # FIXME: any working C++ compiler should be OK
. ./defs || Exit 1
cat >fake-yacc <<'END'
unset YACC || :
cat >> configure.in <<'END'
-AC_PROG_CXX
+AC_SUBST([CXX], [false])
# Simulate presence of Yacc using our fake-yacc script.
AC_SUBST([YACC], ['$(abs_top_srcdir)'/fake-yacc])
AC_OUTPUT
END
cat > Makefile.am <<'END'
+AUTOMAKE_OPTIONS = no-dependencies
bin_PROGRAMS = foo bar
foo_SOURCES = main.cc foo.yy
bar_SOURCES = main.cc bar.y++
grep '\$(YFLAGS).*\$(bar_YFLAGS)' Makefile.in && Exit 1
grep '\$(YFLAGS).*\$(AM_YFLAGS)' Makefile.in && Exit 1
-echo 'int main(void) { return 0; }' > main.cc
: > foo.yy
: > bar.y++
$AUTOCONF
./configure
-env YFLAGS=__user_flags__ $MAKE -e
-
-ls -l
+env YFLAGS=__user_flags__ $MAKE -e foo.cc bar-bar.c++
cat foo.cc
-grep '__am_flags__.*__user_flags__' foo.cc
cat bar-bar.c++
+
+grep '__am_flags__.*__user_flags__' foo.cc
grep '__bar_flags__.*__user_flags__' bar-bar.c++
:
projects / platform / upstream / automake.git / commitdiff