+ 2011-08-22 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: use the new awk+shell TAP driver in our own testsuite
+ * tests/Makefile.am (TAP_LOG_DRIVER): Redefine to use the new
+ awk+shell tap driver, which should be complete enough for use in
+ a "controlled" production environment.
+
+ 2011-08-22 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap/awk: handle non-zero exit status from the test command
+ * lib/tap-driver.sh (get_test_exit_message): New function in the
+ awk script, used to extract the exit status of the test program,
+ or at least a good guess of it.
+ (write_test_results): Use it, reporting an ERROR result if it
+ detects that the test program exited with a non-zero status.
+ * tests/tap-signal.test: Account for the differences in the
+ error messages generated by the awk and perl TAP drivers. Fix
+ an unrelated typo in comments since we are at it.
+
+ 2011-08-22 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap/awk: prepare to fetch the exit status of the test command
+ * lib/tap-driver.sh: Rewrite some logic in the main parsing loop,
+ to make it possible to read the exit status of the test command
+ from the last line of the input stream.
+
+ 2011-08-22 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap/awk: refactor for future changes
+ * lib/tap-driver.sh: Rewrite the awk script so that the parsing
+ of the input stream is implemented as a hand-rolled loop in the
+ BEGIN block, using the `getline' builtin.
+
+ 2011-08-22 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: avoid spurious failure if rst2html program is missing
+ * tests/tap-html.test ($required): Require `rst2html'.
+
+ 2011-08-22 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap/awk: support Solaris /usr/xpg4/bin/awk
+ * lib/tap-driver.sh: Use `\t', not `\\t', to represent tabulation
+ character in regexps, even when inside double-quoted strings; that
+ seems to be more portable to Solaris 10 XPG4 awk.
+
+ 2011-08-22 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap/awk: support Solaris /usr/bin/nawk
+ * lib/tap-driver.sh (close_or_die): Remove, the use of the `close'
+ builtin in there was confusing /usr/bin/nawk from Solaris 10 into
+ a syntax error.
+ (write_test_results): Simply use `close' on the `*.trs' file, that
+ should be enough.
+
+ 2011-08-21 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap/awk: correctly extract message from SKIP plans
+ * lib/tap-driver.sh (/^1\.\.0+[ \t]*#/): Add a forgotten `?' to
+ the regex used to extract the skip message from a "1..0" TAP plan.
+ * tests/planskip.test: Check for the just-fixed bug.
+ * tests/tap-planskip-unplanned-corner.test: Don't be too strict
+ w.r.t. the formatting of the "skip whole test" message, to avoid
+ failing due to bugs irrelevant for this test (like the one just
+ fixed).
+
+ 2011-08-21 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap/awk: support colorized output
+ * lib/tap-driver.sh: Pass definition of colors to the BEGIN block
+ of the awk script, using ...
+ ($init_colors): ... this new variable, which supersedes ...
+ ($red, $grn, $lgn, $blu, $mgn, $std): ... these variables, which
+ have been consequently removed.
+ ($color_tests): We don't have to pass this anymore to the awk
+ script, so do not convert it to a numeric boolean.
+ (decorate_result): Implement for real.
+ With this change, the test 'tap-color.test' now passes.
+
+ 2011-08-21 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap: real (but still incomplete) awk implementation of TAP driver
+ * lib/tap-driver.sh: Add an incomplete, but mostly working,
+ implementation of a TAP parser and driver in awk. It doesn't yet
+ support colorized output, fetching of exit status from test
+ programs, nor a way to escape TAP directives in TAP result lines,
+ but passes all the tests of TAP support in the automake testsuite,
+ apart from the following ones:
+ - tap-color.test
+ - tap-escape-directive.test
+ - tap-exit.test
+ - tap-missing-plan-and-bad-exit.test
+ - tap-passthrough-exit.test
+ - tap-planskip-badexit.test
+ - tap-planskip-unplanned-corner.test
+ - tap-signal.test
+ Tested on Debian GNU/Linux with GNU awk 3.1.7 and 3.0.2, "original
+ awk" 2010-05-23-1, and mawk 1.3.3-15 (with which also the test
+ `tap-realtime.test' fails, in addition to those listed above).
+
+ 2011-08-21 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: fix spurious failure due to Solaris 'wc' incompatibility
+ * tests/tap-number-wordboundary.test ($planned): Strip any extra
+ whitespace from the output of `wc', to cater to Solaris 10.
+
+ 2011-08-21 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ coverage: keyword "SKIP" in TAP plan is case-insensitive
+ * tests/tap-planskip-case-insensitive.test: New test.
+ * tests/tap-planskip.test: Remove now-duplicated checks.
+ * tests/Makefile.am (tap_with_common_setup_tests): Update.
+
+ 2011-08-21 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ coverage: ambiguous use of TAP "TODO"/"SKIP" directives
+ * tests/tap-ambiguous-directive.test: New test.
+ * tests/tap-todo-skip-together.test: Reference it in heading
+ comments.
+ * tests/Makefile.am (tap_with_common_setup_tests): Update.
+
+ 2011-08-21 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ coverage: non-directive comments in TAP results are preserved
+ * tests/tap-result-comment.test: New test.
+ * tests/Makefile.am (tap_with_common_setup_tests): Update.
+
+ 2011-08-20 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ coverage: TAP result number followed by non-whitespace word boundary
+ * tests/tap-number-wordboundary.test: New test.
+ * tests/Makefile.am (tap_with_common_setup_tests): Update.
+
+ 2011-08-20 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ coverage: a TAP result numbered "0" is always out-of-order
+ * tests/tap-test-number-0.test: New, checks that a test result
+ numbered as 0 is to be considered out-of-order; this is
+ consistent with the behaviour of the `prove' utility.
+ * tests/Makefile.am (tap_with_common_setup_tests): Update.
+
+ 2011-08-20 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ cosmetics: fix botched indentation in perl TAP driver
+ * lib/tap-driver.pl (main, Getopt::Long::GetOptions): Fix
+ improper or botched indentation.
+
+ 2011-08-20 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ coverage: some more tests on corner cases of TAP support
+ * tests/tap-no-spurious.test: Extend checks verifying that a line
+ matching, say, the regex "^ok[a-zA-Z0-9_]" is *not* considered a
+ TAP result.
+ * tests/tap-no-spurious-numbers.test: New test, checking that our
+ TAP driver doesn't spuriously recognize as TAP result numbers what
+ is not (even if it seems pretty close).
+ * tests/tap-negative-numbers.test: New test, checking that our
+ TAP driver doesn't spuriously recognize negative TAP result
+ numbers, but correctly interprets them as test descriptions
+ instead.
+ * tests/tap-plan-leading-zero.test, tap-numbers-leading-zero.test:
+ New tests, checking how our driver fares in recognizing numbers
+ with leading zeroes in TAP results or TAP plans.
+ * tests/tap-planskip-malformed.test: New test, checking that a
+ malformed TAP plan is not recognized as a valid plan.
+ * tests/tap-plan-whitespace.test: New test, checking that a TAP
+ plan line with trailing whitespace is recognized and handled
+ correctly.
+ * tests/Makefile.am (tap_with_common_setup_tests): Update.
+
2011-08-19 Stefano Lattarini <stefano.lattarini@gmail.com>
+ testsuite: various fixlets and tweakings
+ * tests/seenc.test: Make grepping of automake stderr stricter.
+ Add a trailing `:' command.
+ * tests/symlink.test: Likewise. Also, define `$am_create_testdir'
+ to "empty" to avoid bringing in unused auxiliary files (only to
+ have to remove them right away), and use proper m4 quoting in
+ `configure.in'.
+ * tests/vpath.test: Make grepping of generated `Makefile.in'
+ slightly stricter. Prefer trailing `:' over trailing `Exit 0'.
+ * tests/suffix6c.test: Unset OBJEXT to avoid interferences from
+ the environment.
+ * tests/suffix12.test: Do not whitespace-indent `##' comments
+ when they are embedded in a makefile rule: having them indented
+ is not part of the Automake API, and might cause failures with
+ e.g., Tru64 make.
+ * tests/syntax.test: Simplify the `Makefile.am' to ensure that
+ automake doesn't fail for the wrong reasons. Make grepping of
+ automake stderr slightly stricter.
+ * tests/test-harness-vpath-rewrite.test: Remove useless variable
+ definition from `Makefile.am'.
+ * tests/test-driver-custom-multitest.test: Fix typo in comments.
+ * tests/self-check-me.tap: Fix minor typo in test description.
+ * tests/vars3.test: Make grepping of automake stderr stricter
+ and safer.
+ * tests/version6.test: Add sanity check, verifying that the
+ version number extracted from `automake --version' output seems
+ legit.
+ * tests/auxdir2.test: Renamed ...
+ * tests/auxdir-compauted.tap: ... to this, and converted to the
+ use of TAP.
+ * tests/auxdir4.test: Renamed ...
+ * tests/auxdir-unportable.tap: ... to this, and converted to the
+ use of TAP.
+ * tests/auxdir3.test: Renamed ...
+ * tests/auxdir-misplaced.test: ... to this.
+ * tests/auxdir5.test: Renamed ...
+ * tests/auxdir-nonexistent.test: ... to this.
+ * tests/auxdir9.test: Renamed ...
+ * tests/auxdir-autodetect.test: ... to this.
+ * tests/Makefile.am (TESTS): Update.
+ (XFAIL_TESTS): Remove `auxdir2.test'.
+
+2011-08-19 Stefano Lattarini <stefano.lattarini@gmail.com>
+
testsuite: fix some redundant autotools calls in tests on TAP
* tests/tap-deps.test: Do not call autotools and configure; there
is no need to, since this script sources `tap-setup.sh'.
top_build_prefix = @top_build_prefix@
top_builddir = @top_builddir@
top_srcdir = @top_srcdir@
-TEST_EXTENSIONS = .test .tap
+
+# Run the tests with the shell detected at configure time.
+LOG_COMPILER = $(SHELL)
+TEST_EXTENSIONS = .test .tap .ptest .shtst
+TEST_LOG_COMPILER = $(LOG_COMPILER)
+TAP_LOG_COMPILER = $(LOG_COMPILER)
- TAP_LOG_DRIVER = $(PERL) $(top_srcdir)/lib/tap-driver.pl
+ TAP_LOG_DRIVER = AM_TAP_AWK='$(AWK)' $(SHELL) $(top_srcdir)/lib/tap-driver.sh
-TAP_LOG_DRIVER_FLAGS = --merge
-MAINTAINERCLEANFILES = $(parallel_tests) $(instspc_tests)
-EXTRA_DIST = ChangeLog-old gen-parallel-tests instspc-tests.sh \
- $(TESTS) tap-functions.sh plain-functions.sh \
+TAP_LOG_DRIVER_FLAGS = --merge --comments --diagnostic-string `printf \
+ '\043%%\043\n'`
+MAINTAINERCLEANFILES =
+EXTRA_DIST = ChangeLog-old gen-parallel-tests parallel-tests.sh \
+ gen-config-shell-tests config-shell-tests.sh \
+ $(handwritten_tests) tap-functions.sh plain-functions.sh \
trivial-test-driver testsuite-summary-checks.sh \
extract-testsuite-summary tap-setup.sh tap-summary-aux.sh
-XFAIL_TESTS = all.test auxdir2.test cond17.test gcj6.test \
- override-conditional-2.test pr8365-remake-timing.test \
- yacc-dist-nobuild-subdir.test txinfo5.test \
- $(instspc_xfail_tests)
-parallel_tests = backcompat5-p.test check-concurrency-bug9245-p.test \
- check-exported-srcdir-p.test check-fd-redirect-p.test \
- check-tests-in-builddir-p.test check-p.test check11-p.test \
- check12-p.test check2-p.test check3-p.test check4-p.test \
- check5-p.test check6-p.test check7-p.test check8-p.test \
- check9-p.test color-p.test color2-p.test comment9-p.test \
- dejagnu-p.test exeext4-p.test maken3-p.test maken4-p.test \
- posixsubst-tests-p.test tests-environment-p.test
-instspc_tests = instspc-squote-build.test instspc-squote-install.test \
- instspc-dquote-build.test instspc-dquote-install.test \
- instspc-bquote-build.test instspc-bquote-install.test \
- instspc-sharp-build.test instspc-sharp-install.test \
- instspc-dollar-build.test instspc-dollar-install.test \
- instspc-bang-build.test instspc-bang-install.test \
- instspc-bslash-build.test instspc-bslash-install.test \
- instspc-ampersand-build.test instspc-ampersand-install.test \
- instspc-percent-build.test instspc-percent-install.test \
- instspc-leftpar-build.test instspc-leftpar-install.test \
- instspc-rightpar-build.test instspc-rightpar-install.test \
- instspc-pipe-build.test instspc-pipe-install.test \
- instspc-caret-build.test instspc-caret-install.test \
- instspc-tilde-build.test instspc-tilde-install.test \
- instspc-qmark-build.test instspc-qmark-install.test \
- instspc-star-build.test instspc-star-install.test \
- instspc-plus-build.test instspc-plus-install.test \
- instspc-minus-build.test instspc-minus-install.test \
- instspc-comma-build.test instspc-comma-install.test \
- instspc-colon-build.test instspc-colon-install.test \
- instspc-semicol-build.test instspc-semicol-install.test \
- instspc-equal-build.test instspc-equal-install.test \
- instspc-less-build.test instspc-less-install.test \
- instspc-more-build.test instspc-more-install.test \
- instspc-at-build.test instspc-at-install.test \
- instspc-lqbrack-build.test instspc-lqbrack-install.test \
- instspc-rqbrack-build.test instspc-rqbrack-install.test \
- instspc-lcbrack-build.test instspc-lcbrack-install.test \
- instspc-rcbrack-build.test instspc-rcbrack-install.test \
- instspc-space-build.test instspc-space-install.test \
- instspc-tab-build.test instspc-tab-install.test \
- instspc-linefeed-build.test instspc-linefeed-install.test \
- instspc-backspace-build.test instspc-backspace-install.test \
- instspc-formfeed-build.test instspc-formfeed-install.test \
- instspc-carriageret-build.test \
- instspc-carriageret-install.test \
- instspc-quadrigraph0-build.test \
- instspc-quadrigraph0-install.test \
- instspc-quadrigraph1-build.test \
- instspc-quadrigraph1-install.test \
- instspc-quadrigraph2-build.test \
- instspc-quadrigraph2-install.test \
- instspc-quadrigraph3-build.test \
- instspc-quadrigraph3-install.test \
- instspc-quadrigraph4-build.test \
- instspc-quadrigraph4-install.test instspc-a_b-build.test \
- instspc-a_b-install.test instspc-a__b-build.test \
- instspc-a__b-install.test instspc-a_lf_b-build.test \
- instspc-a_lf_b-install.test instspc-dotdotdot-build.test \
- instspc-dotdotdot-install.test instspc-dosdrive-build.test \
- instspc-dosdrive-install.test instspc-miscglob1-build.test \
- instspc-miscglob1-install.test instspc-miscglob2-build.test \
- instspc-miscglob2-install.test
-instspc_xfail_tests = instspc-squote-build.test \
- instspc-dquote-build.test instspc-bquote-build.test \
- instspc-sharp-build.test instspc-dollar-build.test \
- instspc-bslash-build.test instspc-ampersand-build.test \
- instspc-linefeed-build.test instspc-quadrigraph0-build.test \
- instspc-a_lf_b-build.test instspc-squote-install.test \
- instspc-dquote-install.test instspc-bquote-install.test \
- instspc-sharp-install.test instspc-dollar-install.test \
- instspc-linefeed-install.test instspc-a_lf_b-install.test
+XFAIL_TESTS = \
+all.test \
+cond17.test \
+gcj6.test \
+override-conditional-2.test \
+pr8365-remake-timing.test \
+yacc-dist-nobuild-subdir.test \
+txinfo5.test
+
+parallel_tests = backcompat5-p.ptest check-concurrency-bug9245-p.ptest \
+ check-exported-srcdir-p.ptest check-fd-redirect-p.ptest \
+ check-subst-prog-p.ptest check-subst-p.ptest \
+ check-tests-in-builddir-p.ptest check-p.ptest check11-p.ptest \
+ check12-p.ptest check2-p.ptest check3-p.ptest check4-p.ptest \
+ check5-p.ptest check6-p.ptest check7-p.ptest check8-p.ptest \
+ color-p.ptest color2-p.ptest comment9-p.ptest dejagnu-p.ptest \
+ exeext4-p.ptest maken3-p.ptest maken4-p.ptest \
+ posixsubst-tests-p.ptest tests-environment-p.ptest
+PTEST_LOG_COMPILER = $(LOG_COMPILER) $(srcdir)/parallel-tests.sh
+config_shell_tests = ar-lib-w.shtst compile-w.shtst compile2-w.shtst \
+ compile3-w.shtst compile4-w.shtst compile5-w.shtst \
+ compile6-w.shtst instsh2-w.shtst instsh3-w.shtst \
+ mdate5-w.shtst mdate6-w.shtst missing-w.shtst missing2-w.shtst \
+ missing3-w.shtst missing5-w.shtst mkinst3-w.shtst
+SHTST_LOG_COMPILER = $(LOG_COMPILER) $(srcdir)/config-shell-tests.sh
# Some testsuite-influential variables should be overridable from the
# test scripts, but not from the environment.
projects / platform / upstream / automake.git / commitdiff