From: Stefano Lattarini Date: Fri, 19 Aug 2011 08:21:01 +0000 (+0200) Subject: Merge branch 'test-protocols' into testsuite-work X-Git-Tag: v1.11b~257^2~99 X-Git-Url: http://review.tizen.org/git/?a=commitdiff_plain;h=82796c70d0a67f9af403fd010add959861c0ba40;p=platform%2Fupstream%2Fautomake.git Merge branch 'test-protocols' into testsuite-work * test-protocols: testsuite: fix some redundant autotools calls in tests on TAP * THANKS (Daniel Richard G.): Update e-mail address. java: avoid compilation errors when CLASSPATH is empty parallel-tests: no more spurious successes for FreeBSD make tap: improve granularity for tests on problematic TAP messages tap: correctly handle string "0" in TAP messages tap: a minor simplification in the perl TAP driver parallel-tests: fix help screen for test driver scripts tap: add a dummy TAP driver script implemented in shell + awk testsuite: refactor tests on TAP support in view of future changes fix: regenerate tests/Makefile.in hacking: we now require autoconf 2.68 --- 82796c70d0a67f9af403fd010add959861c0ba40 diff --cc ChangeLog index eeb8374,f86d1cd..e1d86f0 --- a/ChangeLog +++ b/ChangeLog @@@ -1,9 -1,142 +1,148 @@@ + 2011-08-19 Stefano Lattarini + + testsuite: fix some redundant autotools calls in tests on TAP + * tests/tap-deps.test: Do not call autotools and configure; there + is no need to, since this script sources `tap-setup.sh'. + * tests/tap-empty-diagnostic.test: Likewise. + * tests/tap-escape-directive.test: Likewise. + * tests/tap-fancy2.test: Likewise. + * tests/tap-no-spurious.test: Likewise. + * tests/tap-out-of-order.test: Likewise. + + 2011-08-17 Stefano Lattarini + + * THANKS (Daniel Richard G.): Update e-mail address. + + 2011-08-16 Daniel Richard G. (tiny change) + Stefano Lattarini + + java: avoid compilation errors when CLASSPATH is empty + * lib/am/java.am (CLASSPATH_ENV): When redefining `$CLASSPATH', + do not append an empty component in case the previous value of + CLASSPATH is empty or unset. + * tests/java-empty-classpath.test: New test. + * tests/Makefile.am (TESTS): Update. + Fixes automake bug#9306. + + 2011-08-16 Stefano Lattarini + + parallel-tests: no more spurious successes for FreeBSD make + Work around a bug of FreeBSD make bug that was causing the + automake-generated "check" target to complete with success + even if some tests failed; this happened only when FreeBSD + make was run in concurrent mode (as in, e.g., "make -j2 + check"). The bug is not present in NetBSD make. + This change fixes automake bug#9245: + + See also FreeBSD PR bin/159730: + + * lib/am/check.am [%?PARALLEL_TESTS%] $(TEST_SUITE_LOG): Use a + more "safe" (and apparently redundant) idiom to exit with error, + so that the non-zero exit status is picked up also by FreeBSD + make when it's running in concurrent mode. + * NEWS: Update. + * tests/check-concurrency-bug9245.test: New test. + * tests/Makefile.am (TESTS): Update. + + 2011-08-11 Stefano Lattarini + + hacking: we now require autoconf 2.68 + * HACKING ("Working in git"): Use autoconf and autom4te 2.68, + not 2.67, in the examples. + + 2011-08-17 Stefano Lattarini + + tap: improve granularity for tests on problematic TAP messages + * tests/tap-message-0.test: Break up into ... + * tests/tap-msg0-result.test, tests/tap-msg0-directive.test, + tests/tap-msg0-planskip.test, tests/tap-msg0-misc.test, + tests/tap-msg0-bailout.test: ... these new tests, and extend + a little. + * tests/Makefile.am (tap_with_common_setup_tests): Update. + + 2011-08-17 Stefano Lattarini + + tap: correctly handle string "0" in TAP messages + * lib/tap-driver.pl (is_null_string): New function, can be used + to determine whether a given string variable is empty or undefined. + Useful to avoid pitfalls like: + if ($message) { print "$message\n"; } + which wouldn't print anything if $message is the literal "0". + (handle_tap_test, handle_tap_plan, handle_tap_bailout): Use it, + to avoid missing messages composed only by a literal "0" in TAP + result descriptions and in skip, todo and bailout explanations. + * tests/tap-message-0.test: Enhance. + * tests/Makefile.am (XFAIL_TESTS): Remove it, it passes now. + + 2011-08-17 Stefano Lattarini + + tap: a minor simplification in the perl TAP driver + * lib/tap-driver.pl: The `--disable-hard-errors' option is a + no-op, so just ignore it and its argument. + + 2011-08-17 Stefano Lattarini + + parallel-tests: fix help screen for test driver scripts + * lib/tap-driver.pl ($USAGE): The `--trs-file' option is mandatory + too. + * lib/tap-driver.sh (print_usage): Likewise. + * lib/test-driver (print_usage): Likewise. + ($scriptversion): Update. + + 2011-08-17 Stefano Lattarini + + tap: add a dummy TAP driver script implemented in shell + awk + The user can also now decide which implementation of the TAP driver + to use in the testsuite by defining the `$am_tap_implementation' + variable to either "perl" or "shell". Future enhancements will + allow the testsuite to automatically run the test scripts on TAP + support with both the TAP driver implementations, to improve + coverage. + * tests/defs (fetch_tap_driver): Honor the `$am_tap_implementation' + variable to decide which implementation of the TAP driver to fetch. + ($am_tap_implementation): Default to "perl". + * tests/tap-common-setup.test: Do not fetch the TAP driver, the + code in tap-setup.sh does that already (and respecting runtime + overriding of `$am_tap_implementation'). + * lib/tap-driver: Renamed ... + * lib/tap-driver.pl: ... to this, and ... + ($ME): ... adjusted this. + * doc/automake.texi: Adjust to the renaming. + * tests/Makefile.am (TAP_LOG_DRIVER): Likewise. + * tests/tap-doc2.test: Likewise. + * lib/tap-driver.sh: New script, still mostly dummy. + * lib/Makefile.am (dist_script_DATA): Update, and since we are at + it, rewrite it to make it easier to add new entries in the future. + + 2011-08-17 Stefano Lattarini + + testsuite: refactor tests on TAP support in view of future changes + * tests/defs (fetch_tap_driver): New subroutine; it fetches the + automake-provided TAP driver from the `lib/' directory into the + current directory, and edits its shebang line so that it will be + run with the perl interpreter determined at configure time. + * tests/tap-setup.sh: Use it. + * tests/tap-common-setup.test: There's no need to AC_SUBST `PERL' + anymore, nor to use it in the Makefile to run the TAP driver. + Also, use the `fetch_tap_driver' function instead of copying the + `tap-driver' auxiliary script directly. + * tests/tap-bad-prog.tap: Likewise. + * tests/tap-diagnostic-custom.test: Likewise. + * tests/tap-doc.test: Likewise. + * tests/tap-merge-stdout-stderr.test: Likewise. + * tests/tap-more.test: Likewise. + * tests/tap-more2.test: Likewise. + * tests/tap-recheck.test: Likewise. + * tests/tap-summary-aux.sh: Likewise. + * tests/tap-basic.test: Likewise, and fix a grammaro in comments + since we are at it. + +2011-08-17 Stefano Lattarini + + test defs: fix typo in comments + * tests/defs (get_shell_script): Fix typo in the function + description. + 2011-08-14 Stefano Lattarini coverage: missing tap plan and non-zero exit status diff --cc tests/Makefile.am index 0db204e,2d630a9..b03d6a6 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@@ -16,24 -16,9 +16,24 @@@ # You should have received a copy of the GNU General Public License # along with this program. If not, see . +# Run the tests with the shell detected at configure time. +LOG_COMPILER = $(SHELL) + TEST_EXTENSIONS = .test .tap +TEST_LOG_COMPILER = $(LOG_COMPILER) +TAP_LOG_COMPILER = $(LOG_COMPILER) + - TAP_LOG_DRIVER = $(PERL) $(top_srcdir)/lib/tap-driver + TAP_LOG_DRIVER = $(PERL) $(top_srcdir)/lib/tap-driver.pl -TAP_LOG_DRIVER_FLAGS = --merge +TAP_LOG_DRIVER_FLAGS = --merge --comments +## Avoid spurious TAP diagnostic. We use an awful indirection with a +## shell command substitution here, since the string needs to contain +## `#' characters, and these can't be escaped portably in a make macro +## definition. +## FIXME: this is quite inefficient though, as it adds one extra fork +## FIXME: per TAP test script; this is OK for the moment, since we have +## FIXME: few such scripts, but might become an issue in the future. +## Keep this in sync with the definition of $diag_string_ in tests/defs. +TAP_LOG_DRIVER_FLAGS += --diagnostic-string `printf '\043%%\043\n'` MAINTAINERCLEANFILES = EXTRA_DIST = ChangeLog-old diff --cc tests/Makefile.in index 5f50e7d,dc7cb80..1dfcffd --- a/tests/Makefile.in +++ b/tests/Makefile.in @@@ -292,48 -278,88 +292,47 @@@ target_alias = @target_alias top_build_prefix = @top_build_prefix@ top_builddir = @top_builddir@ top_srcdir = @top_srcdir@ -TEST_EXTENSIONS = .test .tap + +# Run the tests with the shell detected at configure time. +LOG_COMPILER = $(SHELL) +TEST_EXTENSIONS = .test .tap .ptest .shtst +TEST_LOG_COMPILER = $(LOG_COMPILER) +TAP_LOG_COMPILER = $(LOG_COMPILER) - TAP_LOG_DRIVER = $(PERL) $(top_srcdir)/lib/tap-driver + TAP_LOG_DRIVER = $(PERL) $(top_srcdir)/lib/tap-driver.pl -TAP_LOG_DRIVER_FLAGS = --merge -MAINTAINERCLEANFILES = $(parallel_tests) $(instspc_tests) -EXTRA_DIST = ChangeLog-old gen-parallel-tests instspc-tests.sh \ - $(TESTS) tap-functions.sh plain-functions.sh \ +TAP_LOG_DRIVER_FLAGS = --merge --comments --diagnostic-string `printf \ + '\043%%\043\n'` +MAINTAINERCLEANFILES = +EXTRA_DIST = ChangeLog-old gen-parallel-tests parallel-tests.sh \ + gen-config-shell-tests config-shell-tests.sh \ + $(handwritten_tests) tap-functions.sh plain-functions.sh \ trivial-test-driver testsuite-summary-checks.sh \ extract-testsuite-summary tap-setup.sh tap-summary-aux.sh -XFAIL_TESTS = all.test auxdir2.test cond17.test gcj6.test \ - override-conditional-2.test pr8365-remake-timing.test \ - yacc-dist-nobuild-subdir.test txinfo5.test \ - $(instspc_xfail_tests) -parallel_tests = backcompat5-p.test check-concurrency-bug9245-p.test \ - check-exported-srcdir-p.test check-fd-redirect-p.test \ - check-tests-in-builddir-p.test check-p.test check11-p.test \ - check12-p.test check2-p.test check3-p.test check4-p.test \ - check5-p.test check6-p.test check7-p.test check8-p.test \ - check9-p.test color-p.test color2-p.test comment9-p.test \ - dejagnu-p.test exeext4-p.test maken3-p.test maken4-p.test \ - posixsubst-tests-p.test tests-environment-p.test -instspc_tests = instspc-squote-build.test instspc-squote-install.test \ - instspc-dquote-build.test instspc-dquote-install.test \ - instspc-bquote-build.test instspc-bquote-install.test \ - instspc-sharp-build.test instspc-sharp-install.test \ - instspc-dollar-build.test instspc-dollar-install.test \ - instspc-bang-build.test instspc-bang-install.test \ - instspc-bslash-build.test instspc-bslash-install.test \ - instspc-ampersand-build.test instspc-ampersand-install.test \ - instspc-percent-build.test instspc-percent-install.test \ - instspc-leftpar-build.test instspc-leftpar-install.test \ - instspc-rightpar-build.test instspc-rightpar-install.test \ - instspc-pipe-build.test instspc-pipe-install.test \ - instspc-caret-build.test instspc-caret-install.test \ - instspc-tilde-build.test instspc-tilde-install.test \ - instspc-qmark-build.test instspc-qmark-install.test \ - instspc-star-build.test instspc-star-install.test \ - instspc-plus-build.test instspc-plus-install.test \ - instspc-minus-build.test instspc-minus-install.test \ - instspc-comma-build.test instspc-comma-install.test \ - instspc-colon-build.test instspc-colon-install.test \ - instspc-semicol-build.test instspc-semicol-install.test \ - instspc-equal-build.test instspc-equal-install.test \ - instspc-less-build.test instspc-less-install.test \ - instspc-more-build.test instspc-more-install.test \ - instspc-at-build.test instspc-at-install.test \ - instspc-lqbrack-build.test instspc-lqbrack-install.test \ - instspc-rqbrack-build.test instspc-rqbrack-install.test \ - instspc-lcbrack-build.test instspc-lcbrack-install.test \ - instspc-rcbrack-build.test instspc-rcbrack-install.test \ - instspc-space-build.test instspc-space-install.test \ - instspc-tab-build.test instspc-tab-install.test \ - instspc-linefeed-build.test instspc-linefeed-install.test \ - instspc-backspace-build.test instspc-backspace-install.test \ - instspc-formfeed-build.test instspc-formfeed-install.test \ - instspc-carriageret-build.test \ - instspc-carriageret-install.test \ - instspc-quadrigraph0-build.test \ - instspc-quadrigraph0-install.test \ - instspc-quadrigraph1-build.test \ - instspc-quadrigraph1-install.test \ - instspc-quadrigraph2-build.test \ - instspc-quadrigraph2-install.test \ - instspc-quadrigraph3-build.test \ - instspc-quadrigraph3-install.test \ - instspc-quadrigraph4-build.test \ - instspc-quadrigraph4-install.test instspc-a_b-build.test \ - instspc-a_b-install.test instspc-a__b-build.test \ - instspc-a__b-install.test instspc-a_lf_b-build.test \ - instspc-a_lf_b-install.test instspc-dotdotdot-build.test \ - instspc-dotdotdot-install.test instspc-dosdrive-build.test \ - instspc-dosdrive-install.test instspc-miscglob1-build.test \ - instspc-miscglob1-install.test instspc-miscglob2-build.test \ - instspc-miscglob2-install.test -instspc_xfail_tests = instspc-squote-build.test \ - instspc-dquote-build.test instspc-bquote-build.test \ - instspc-sharp-build.test instspc-dollar-build.test \ - instspc-bslash-build.test instspc-ampersand-build.test \ - instspc-linefeed-build.test instspc-quadrigraph0-build.test \ - instspc-a_lf_b-build.test instspc-squote-install.test \ - instspc-dquote-install.test instspc-bquote-install.test \ - instspc-sharp-install.test instspc-dollar-install.test \ - instspc-linefeed-install.test instspc-a_lf_b-install.test +XFAIL_TESTS = \ +all.test \ +auxdir2.test \ +cond17.test \ +gcj6.test \ +override-conditional-2.test \ +pr8365-remake-timing.test \ +yacc-dist-nobuild-subdir.test \ - tap-message-0.test \ +txinfo5.test + - parallel_tests = backcompat5-p.ptest check-exported-srcdir-p.ptest \ - check-fd-redirect-p.ptest check-subst-prog-p.ptest \ - check-subst-p.ptest check-tests-in-builddir-p.ptest \ - check-p.ptest check11-p.ptest check12-p.ptest check2-p.ptest \ - check3-p.ptest check4-p.ptest check5-p.ptest check6-p.ptest \ - check7-p.ptest check8-p.ptest color-p.ptest color2-p.ptest \ - comment9-p.ptest dejagnu-p.ptest exeext4-p.ptest \ - maken3-p.ptest maken4-p.ptest posixsubst-tests-p.ptest \ - tests-environment-p.ptest ++parallel_tests = backcompat5-p.ptest check-concurrency-bug9245-p.ptest \ ++ check-exported-srcdir-p.ptest check-fd-redirect-p.ptest \ ++ check-subst-prog-p.ptest check-subst-p.ptest \ ++ check-tests-in-builddir-p.ptest check-p.ptest check11-p.ptest \ ++ check12-p.ptest check2-p.ptest check3-p.ptest check4-p.ptest \ ++ check5-p.ptest check6-p.ptest check7-p.ptest check8-p.ptest \ ++ color-p.ptest color2-p.ptest comment9-p.ptest dejagnu-p.ptest \ ++ exeext4-p.ptest maken3-p.ptest maken4-p.ptest \ ++ posixsubst-tests-p.ptest tests-environment-p.ptest +PTEST_LOG_COMPILER = $(LOG_COMPILER) $(srcdir)/parallel-tests.sh +config_shell_tests = ar-lib-w.shtst compile-w.shtst compile2-w.shtst \ + compile3-w.shtst compile4-w.shtst compile5-w.shtst \ + compile6-w.shtst instsh2-w.shtst instsh3-w.shtst \ + mdate5-w.shtst mdate6-w.shtst missing-w.shtst missing2-w.shtst \ + missing3-w.shtst missing5-w.shtst mkinst3-w.shtst +SHTST_LOG_COMPILER = $(LOG_COMPILER) $(srcdir)/config-shell-tests.sh # Some testsuite-influential variables should be overridable from the # test scripts, but not from the environment. @@@ -1859,10 -1852,9 +1863,11 @@@ uninstall-am uninstall uninstall-am backcompat5-p.log: backcompat5.test + check-concurrency-bug9245-p.log: check-concurrency-bug9245.test check-exported-srcdir-p.log: check-exported-srcdir.test check-fd-redirect-p.log: check-fd-redirect.test +check-subst-prog-p.log: check-subst-prog.test +check-subst-p.log: check-subst.test check-tests-in-builddir-p.log: check-tests-in-builddir.test check-p.log: check.test check11-p.log: check11.test diff --cc tests/defs index 87c9c82,4d21a49..e4e1840 --- a/tests/defs +++ b/tests/defs @@@ -452,41 -306,30 +452,65 @@@ unindent ( } sed_unindent_prog="" # Avoid interferences from the environment. +# get_shell_script SCRIPT-NAME +# ----------------------------- +# Fetch an Automake-provided shell script from the `lib/' directory into +# the current directory, and, if the `$test_prefer_config_shell' variable +# is set to "yes", modify its shebang line to use $SHELL instead of +# /bin/sh. +get_shell_script () +{ + if test x"$test_prefer_config_shell" = x"yes"; then + sed "1s|#!.*|#! $SHELL|" "$top_testsrcdir/lib/$1" > "$1" + chmod a+x "$1" + else + cp "$top_testsrcdir/lib/$1" . + fi + sed 10q "$1" # For debugging. +} + +# require_xsi SHELL +# ----------------- +# Skip the test if the given shell fails to support common XSI constructs. +require_xsi () +{ + test $# -eq 1 || fatal_ "require_xsi needs exactly one argument" + echo "$me: trying some XSI constructs with $1" + $1 -c "$xsi_shell_code" || skip_all_ "$1 lacks XSI features" +} +# Shell code supposed to work only with XSI shells. Keep this in sync +# with libtool.m4:_LT_CHECK_SHELL_FEATURES. +xsi_shell_code=' + _lt_dummy="a/b/c" + test "${_lt_dummy##*/},${_lt_dummy%/*},${_lt_dummy#??}"${_lt_dummy%"$_lt_dummy"}, \ + = c,a/b,b/c, \ + && eval '\''test $(( 1 + 1 )) -eq 2 \ + && test "${#_lt_dummy}" -eq 5'\' + + # fetch_tap_driver + # ---------------- + # Fetch the Automake-provided TAP driver from the `lib/' directory into + # the current directory, and edit its shebang line so that it will be + # run with the perl interpreter determined at configure time. + fetch_tap_driver () + { + # TODO: we should devise a way to make the shell TAP driver tested also + # TODO: with /bin/sh, for better coverage. + case $am_tap_implementation in + perl) + sed "1s|#!.*|#! $PERL -w|" "$top_testsrcdir"/lib/tap-driver.pl ;; + shell) + sed "1s|#!.*|#! $SHELL|" "$top_testsrcdir"/lib/tap-driver.sh ;; + *) + fatal_ "invalid \$am_tap_implementation '$am_tap_implementation'" ;; + esac > tap-driver \ + && chmod a+x tap-driver \ + || framework_failure_ "couldn't fetch $am_tap_implementation TAP driver" + sed 10q tap-driver # For debugging. + } + # The shell/awk implementation of the TAP driver is still mostly dummy, so + # use the perl implementation by default for the moment. + am_tap_implementation=${am_tap_implementation-perl} ## ----------------------------------------------------------- ## ## Checks for required tools, and additional setups (if any) ## diff --cc tests/tap-bad-prog.tap index 8ba153f,c5d3733..1ced399 --- a/tests/tap-bad-prog.tap +++ b/tests/tap-bad-prog.tap @@@ -21,13 -21,9 +21,11 @@@ parallel_tests=yes . ./defs || Exit 1 +plan_ 5 + - cp "$top_testsrcdir"/lib/tap-driver . \ - || fatal_ "failed to fetch auxiliary script tap-driver" + fetch_tap_driver cat >> configure.in < Makefile.am <<'END' - TEST_LOG_DRIVER = $(PERL) $(srcdir)/build-aux/tap-driver + TEST_LOG_DRIVER = $(PERL) $(srcdir)/build-aux/tap-driver.pl TESTS = foo.test bar.test baz.test EXTRA_DIST = $(TESTS) END