2011-08-08 Stefano Lattarini <stefano.lattarini@gmail.com>
+ tests: fix spurious failure with Solaris make
+ * tests/distcheck-configure-flags-am.test: Avoid using `+=' too
+ liberally with AM_DISTCHECK_CONFIGURE_FLAGS, since the line breaks
+ so introduced, in conjunction with single quotes, might confuse
+ Solaris make.
+
++2011-08-08 Stefano Lattarini <stefano.lattarini@gmail.com>
++
+ test harness: fixlet in workaround for VPATH rewrite issue
+ Fixes a minor in previous commit `v1.11-983-gfda3de5'.
+ * lib/am/check.am (am__set_TESTS_bases): Add forgotten end-of-line
+ anchor to sed expression. In comments, add reference to ...
+ * tests/test-trs-basic.test: ... this test, which has been adapted
+ and tightened (and tweaked).
+
+2011-08-08 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ test harness: work around a VPATH rewrite issue
+ * lib/am/check.am (am__TEST_BASES): Removed, it's role taken
+ over by ...
+ (am__set_TESTS_bases): ... these new variable.
+ ($(TEST_SUITE_LOG): Use it, to avoid VPATH rewrite issues.
+ * automake.in (handle_tests): Update the code for the cleanup
+ of the `.trs' file to use `$(TEST_LOGS)' instead of
+ `$(am__TEST_BASES)'.
+ * tests/test-harness-vpath-rewrite.test: New test, better
+ exposing the problem fixed by this change
+ * tests/Makefile.am (TESTS): Add it.
+ * tests/test-trs-basic.test: Update and extend.
+
+2011-08-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ parallel-tests: work around Solaris XPG4 make segfault
+ Issue exposed by test `posixsubst-tests-p.test', and similar to
+ the problem solved by commit `v1.11-159-ge7aa360'.
+ * lib/am/check.am [%?PARALLEL_TESTS%] (check-TESTS): Trim trailing
+ whitespace from $list, to avoid triggering a nasty bug (potential
+ segfault) on Solaris XPG4 make and Heirloom make.
+
+2011-08-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: fix weird spurious failure with Solaris /bin/sh
+ Solaris /bin/sh, when killed with a SIGTERM or SIGINT signal, can
+ apparently end up exiting with exit status 208, instead of leaving
+ the correct wide exit status to the parent. See:
+ <http://dbaspot.com/shell/396118-bourne-shell-exit-code-term.html>
+ Work around this incompatibility.
+ * tap-signal.test: Look for the above Solaris bug.
+ (signal_caught): Adapt to handle it.
+
+2011-08-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: fix another spurious failure on Solaris make
+ * tests/parallel-tests-log-override-recheck.test: Filter make
+ output before grepping it, for make implementations that, like
+ Solaris' one, print the whole of the failed recipe on failure.
+
+2011-08-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: fix two spurious failures on Solaris make
+ * tests/tests-environment-and-log-compiler.test: Do not fail if
+ "make -n check" fails, as that is issued only for debugging, its
+ semantics being tested in other test scripts.
+ * tests/test-driver-strip-vpath.test: Likewise.
+
+2011-08-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap driver: refactor and remove dead code
+ * lib/tap-driver (get_tap_line): Removed, ...
+ (all callers): ... updated to use $parser->next directly instead.
+ (peek_tap_line, unget_tap_line, @tap_lines): Remove, they're not
+ used anymore.
+ (TAP_PEEKING): Block deleted, all its subroutines and variables
+ having been removed.
+
+2011-08-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: fix spurious failure in a test on TAP support
+ * tests/tap-passthrough-exit.test: When the `--ignore-exit' flag
+ of the TAP driver is used, don't look for a message reporting the
+ non-zero exit statuses of tests in the log files; such message is
+ not expected to be there anymore. Related simplifications.
+
+2011-08-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap: plan location is more liberal w.r.t. non-TAP lines
+ With this change, only lines that are TAP results will matter
+ w.r.t. the position of the TAP plan in the input; for example,
+ this input:
+ this is a non-TAP line
+ # and this a TAP diagnostic line
+ 1..1
+ ok 1
+ was considered to be an error, diagnosed with a message "test
+ plan in middle of output"; as effect of the current change, such
+ input is now valid. This is more consistent with the behaviour
+ of the `prove' utility.
+ * lib/tap-driver ($lineno): Removed, no more needed.
+ ($tap_stopped): New global variable.
+ (stringify_test_result): Return "ERROR" if a TAP result is found
+ when `$tap_stopped' is set to true.
+ (handle_tap_test): Diagnose TAP results that comes after a late
+ plan. Add a couple of blank lines, for clarity.
+ (handle_tap_plan): Set `$tap_stopped' to true after a late plan
+ is encountered. Do not complain anymore for extra non-TAP lines
+ preceding or following the plan. Adjust comments.
+ (main): Don't increment $lineno anymore.
+ * tests/tap-plan.test: Extend a bit, and remove stale comment.
+ * tests/tap-color.test: Adjust.
+ * tests/tap-passthrough.test: Likewise.
+ * tests/tap-plan-corner.test: Adjust and extend.
+ * tests/tap-plan-errors.test: Likewise.
+ * tests/tap-plan-middle.test: New test.
+ * tests/tap-plan-corner2.test: Delete, it's obsolete now.
+ * tests/Makefile.am (XFAIL_TESTS): Remove it.
+ (tap_with_common_setup_tests): Likewise, and add
+ `tap-plan-corner.test'.
+
+2011-08-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: remove now-passing test from XFAIL_TESTS
+ * tests/Makefile.am (XFAIL_TESTS): Remove `tap-signal.test',
+ which is passing since previous commit `v1.11-974-gc7fa872'.
+
+2011-08-06 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap driver: handle signals received by the tests being run
+ * lib/tap-driver (get_test_exit_message): Also deal with signals,
+ by using the `wait' method of the TAP::Parser object instead of
+ the `exit' method. This required the use of the standard perl
+ module `POSIX'.
+ * doc/automake.texi (Use TAP with the Automake test harness):
+ Document that `--ignore-exit' has effect also on terminating
+ signals. Add a "synchronizing" comment that references the tests
+ 'tap-exit.test' and 'tap-signal.test'.
+ * tests/tap-signal.test: Extend and adjust.
+
+2011-08-06 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ test driver: a preparatory refactoring (2)
+ * lib/tap-driver (finish): Move code to fetch the message about
+ the exit status of the test being run to ...
+ (get_test_exit_message): ... this new subroutine.
+
+2011-08-06 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap driver: a preparatory refactoring (1)
+ * lib/tap-driver (finish): Do not flush the TAP stream nor
+ fetch the exit status of the TAP producer if running with
+ the `--ignore-exit' option. Obviously, don't try to write
+ such exit status in the test log anymore.
+
+2011-08-06 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ cosmetics: typofix in the name of a function in the TAP driver
+ * lib/tap-driver (testuite_error): Renamed ...
+ (testsuite_error): ... to this, and minor related reformatting.
+
+2011-08-06 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap: scripts with a SKIP plan but with exit status != 0 must error
+ This change has been motivated by Automake's own testsuite. Some
+ TAP tests there were erroring out (due to unexpected, unhandled
+ failures) before having encountered TAP result, so that the
+ simple-minded shell library implementing TAP generation ended up
+ printing a "1..0" trailing test plan; this caused the script to be
+ reported as a SKIP rather than an ERROR -- a nasty false negative.
+ * lib/tap-driver: Add prototypes for each subroutine, to free up
+ the order in which they can be defined and called.
+ (main): Move the code checking for a bad exit status of the TAP
+ producer ...
+ (finish): ... here, and flush the TAP stream to ensure that the
+ parser always obtains the producer's exit status.
+ * tests/tap-skip-whole-badexit.test: New test.
+ * tests/Makefile.am (tap_with_common_setup_tests): Add it.
+
+2011-08-06 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap: fix whitespace munging of diagnostic messages
+ * lib/tap-driver (extract_tap_comment): Pass the `g' flag to the
+ substitution operator, to strip also trailing whitespaces. Fixes
+ a failure in test `tap-whitespace-normalization.test'.
+
+2011-08-05 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap: fix typo in TAP driver version message
+ * lib/tap-driver (anonymous subroutine printing the script version
+ message): Add missing trailing newline to the version message.
+
+2011-08-05 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap docs: we don't support pragma or version directives (yet)
+ * doc/automake.texi (Incompatibilities with other TAP parsers
+ and drivers): Document that limitation. Normalize an unrelated
+ "TODO" item.
+
+2011-08-05 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap: new option to change the string designating TAP diagnostic
+ Here we introduce a new option `--diagnostic-string' in our TAP
+ test driver, that allows the user to specify which string should
+ denote the beginning of a TAP diagnostic line. This change is
+ not gratuitous, nor result if over-engineering: it is motivated
+ by real issues that have emerged during the use of TAP in the
+ Automake's own testsuite (see the commit `v1.11-1082-g9b967c2'
+ "testsuite: yet more use of TAP, and related extensions").
+ * doc/automake.texi (Use TAP with Automake test harness): Document
+ the new option.
+ (Incompatibilities with other TAP parsers and drivers): Report it
+ as a potential source of incompatibility.
+ * lib/tap-driver ($diag_string): New global variable, defaulting
+ to "#", and whose value can be changed ...
+ (Getopt::Long::GetOptions): ... by the newly recognized option
+ `--diagnostic-string'.
+ (handle_tap_comment): Subroutine removed, some of its simple logic
+ inlined ...
+ (main): ... in here, where now ...
+ (extract_tap_comment): ... this new subroutine is used.
+ ($USAGE): Adjust.
+ * tests/tap-diagnostic.test: Make one check slightly stricter.
+ * tests/tap-diag-custom.test: New test.
+ * tests/Makefile.am (tap_other_tests): Add it.
+
+2011-08-05 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ check: add small "synchronization" comment to `test-driver'
+ * lib/test-driver: Add comment to the code initializing ANSI color
+ escapes, telling to keep it in sync with the similar initialization
+ in `lib/am/check.am'.
+
+2011-08-05 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ maintcheck: fix spurious failure w.r.t. use of bare "exit"
+ * tests/check12.test: Cosmetic changes to avoid triggering a
+ spurious failure of the `sc_tests_Exit_not_exit' maintainer
+ check.
+
+2011-08-05 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ check: update comments to reflect recent heavy changes
+ * lib/am/check.am [%?PARALLEL_TESTS%]: Update the comments to
+ reflect the recent changes. Remove quite many comments that
+ were merely duplicating excerpts from the Automake manual.
+
+2011-08-05 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ check: small cleanups and refactorings in test harness and drivers
+ * lib/tap-driver (yn): New subroutine, converts a boolean value to
+ a "yes" or "no" string.
+ (write_test_results): Use it, and related reformatting.
+ * lib/test-driver (trap): Also remove the `.trs' file on signals,
+ for extra safety.
+ (fatal): Remove this function, it's never used.
+ ($logfile, $trsfile): Renamed ...
+ ($log_file, $trs_file): ... to these, for clarity, and in order to
+ be more consistent with the `tap-driver' script.
+ Improve a couple of comments.
+ * automake.in (handle_tests): Don't define anymore the now-obsolete
+ make macro `$(TEST_LOGS_TMP)', nor add it to the list of files to
+ be removed upon "make mostlyclean".
+ * lib/am/check.am ($(TEST_SUITE_LOG)): New shell function `f_ok',
+ tells whether a path refers to an existing, regular, readable file.
+ Use it throughout.
+ (recheck): Be safer w.r.t. make implementation that run recipes
+ with `errexit' shell flag active.
+
+2011-08-05 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: use AM_TESTS_FD_REDIRECT where appropriate
+ * parallel-tests-interrupt.test: Use AM_TESTS_FD_REDIRECT, not
+ AM_TESTS_ENVIRONMENT, to define file descriptors' redirection
+ for "make check". Fix botched comment.
+
+2011-08-05 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ docs: some fixlets in documentation on testsuites support
+ * doc/automake.texi (Script-based Testsuites): Rename node ...
+ (Scripts-based Testsuites): ... to this. Break overly long lines
+ in an example (were causing problems with PDF output).
+ (Parallel Test Harness): Don't intend paragraphs that come after
+ examples or bulleted list.
+ (Overview of Custom Test Drivers Support): Fix typo s/the the/the/.
+ (Declaring Custom Test Drivers): When giving an aside, prefer comma
+ to parentheses, as it disrupts the reading flow less.
+ (API for Custom Test Drivers): Remove a sentence that was basically
+ duplicated from the previous subsection.
+ (Command-line arguments for test drivers): Don't intend paragraph
+ coming after an itemized list. Clarify wording and fix grammaros.
+ Add a couple of cross-references to earlier explanations of `.log'
+ and `.trs' files.
+ (Links and external resources): Node renamed ...
+ (Links and external resources on TAP): ... to this, for clarity.
+ Slightly extend and clarify introductory sentence.
+
+2011-08-04 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ test harness: be aware of more metadata, simplify test drivers
+ * lib/am/check.am ($(TEST_SUITE_LOG)): When producing the
+ global test log, take into account the new metadata field
+ `:global-test-result:, to write nicely formatted RST section
+ titles, instead of leaving this chore to the individual test
+ drivers.
+ (am_rst_section): Re-introduce this variable, as removed in
+ commit `v1.11-870-ga27c9c4'.
+ * lib/test-driver, lib/tap-driver: Write the global test result
+ as metadata in the `.trs' file, not as (part of) a formatted RST
+ subsection title in the `.log' file. Related simplifications;
+ in particular, get rid of temporary files usage.
+ * doc/automake.texi (Log files generation and test results
+ recording): Document the new metadata.
+ * tests/check12.test: Update.
+ * tests/parallel-tests-harderror.test: Likewise.
+ * tests/parallel-tests-interrupt.test: Likewise.
+ * tests/parallel-tests-log-override-2.test: Likewise.
+ * tests/parallel-tests-log-override-recheck.test: Likewise.
+ * tests/parallel-tests-unreadable.test: Likewise.
+ * tests/tap-global-result.test: Likewise.
+ * tests/test-metadata-results.test: Likewise.
+ * tests/test-log.test: Likewise, and another minor unrelated fix.
+ * tests/test-metadata-global-result.test: New test.
+ * tests/Makefile.am (TESTS): Update.
+
+2011-08-03 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tap: check that also a trailing TAP plan can hold a skip directive
+ * tests/tap-skip-whole-lastline.test: New test.
+ * tests/Makefile.am (tap_with_common_setup_tests): Add it.
+
+2011-08-03 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ maintcheck: fix more maintainer-check failures
+ * tests/test-driver-custom-no-html.test (no-rst): Use `EOF',
+ not `EoL', as the here-document delimiter.
+ * tests/test-trs-basic.test: Use `cat + here-doc' rather
+ than `echo' when creating the dummy test scripts, to please
+ maintainer-check.
+ * tests/test-trs-recover.test: Use creative quoting where
+ needed, to please maintainer-check.
+ * tests/parallel-tests-no-color-in-log.test: Likewise.
+ * tests/parallel-tests-dry-run.test: Likewise.
+
2011-08-03 Stefano Lattarini <stefano.lattarini@gmail.com>
maintcheck: fix maintainer-check failures, both real and spurious
projects / platform / upstream / automake.git / commitdiff