Merge branch 'master' into test-protocols
authorStefano Lattarini <stefano.lattarini@gmail.com>
Wed, 29 Jun 2011 08:06:20 +0000 (10:06 +0200)
committerStefano Lattarini <stefano.lattarini@gmail.com>
Wed, 29 Jun 2011 08:06:20 +0000 (10:06 +0200)
* master:
  docs: explain why AM_TESTS_ENVIRONMENT must be semicolon-terminated
  docs: fix unportable example of AM_TESTS_ENVIRONMENT usage
  docs: avoid a footnote, some related rewordings and improvements
  Revert "docs: parallel-tests is not experimental anymore"
  docs: minor cosmetic fixes
  help: improve text about automatically-distributed files
  refactor: split 'usage' subroutine in automake
  tests: fix bug in 'autodist.test'
  parallel-tests: stricter checks on DISABLE_HARD_ERRORS support
  docs: parallel-tests is not experimental anymore
  check: document and test $(TEST_SUITE_LOG) overriding

1  2 
ChangeLog
automake.in
doc/automake.texi
tests/Makefile.am
tests/Makefile.in

diff --cc ChangeLog
index d6dbd13413a248f0819cf270a538e73581d0c347,14364825b6f502b73cb639b970afdcf83a030f24..c99944af6f3058a046ec2c519f7197db62dacbf2
+++ b/ChangeLog
+ 2011-06-29   Stefano Lattarini  <stefano.lattarini@gmail.com>
+       docs: explain why AM_TESTS_ENVIRONMENT must be semicolon-terminated
+       * doc/automake.texi (Simple Tests using parallel-tests): Ditto, and
+       related adjustments.
+       Suggestion by Ralf Wildenhues.
+ 2011-06-29  Stefano Lattarini  <stefano.lattarini@gmail.com>
+       docs: fix unportable example of AM_TESTS_ENVIRONMENT usage
+       * doc/automake.texi (Simple Tests using parallel-tests): The
+       old example on AM_TESTS_ENVIRONMENT relied on unportable shell
+       features, and in particular didn't work with various Korn
+       Shells (see also commit `v1.11-925-g29ca903').  Give another
+       example, simpler this time, but still inspired to real-world
+       usage (the GNU coreutils testsuite).
+ 2011-06-23   Stefano Lattarini  <stefano.lattarini@gmail.com>
+       docs: avoid a footnote, some related rewordings and improvements
+       * doc/automake.texi (Dist): Reword the part about automatically
+       distributed files to avoid a footnote.  Since we are at it, extend
+       a bit, and add an example and a reference to a relevant test case.
+ 2011-06-23   Stefano Lattarini  <stefano.lattarini@gmail.com>
+       docs: minor cosmetic fixes
+       * doc/automake.texi: Break few overly long lines, throughout the
+       file.
+       ("Simple Tests"): Move @vindex for XFAIL_TESTS to the correct
+       position, i.e., before and not after the paragraph where it is
+       introduced.
+       ("Options" @item ansi2knr): Use @pxref instead of @xref.  This
+       fixes a texinfo warning.
+       ("Other things Automake recognizes" @item AM_C_PROTOTYPES): Use
+       @pxref instead of @ref.
+ 2011-06-23   Stefano Lattarini  <stefano.lattarini@gmail.com>
+       help: improve text about automatically-distributed files
+       This change fixes automake bug#7819.
+       * automake.in (usage): Distinguish between files that are always
+       automatically distributed when found, and those which are only
+       "under certain conditions".
+       * doc/automake.texi (Basics of Distribution): Update accordingly.
+       * tests/autodist-subdir.test: Update.
+       * tests/autodist-no-duplicate.test: Likewise.
+       * tests/autodist.test: Likewise.
+       (configure.in): Remove useless call to AM_MAINTAINER_MODE.
+ 2011-06-23   Stefano Lattarini  <stefano.lattarini@gmail.com>
+       refactor: split 'usage' subroutine in automake
+       This change is related to automake bug#7819.
+       * automake.in (print_autodist_files): New subroutine,
+       extracted from ...
+       (usage): ... this, which now uses it.
+       * tests/autodist-no-duplicate.test: New test.
+       * tests/Makefile.am (TESTS): Update.
+ 2011-06-23  Stefano Lattarini  <stefano.lattarini@gmail.com>
+       tests: fix bug in 'autodist.test'
+       * tests/autodist.test: Avoid spurious failure due to no
+       `defs-static' file being found in the parent directory.
 +2011-06-21  Stefano Lattarini  <stefano.lattarini@gmail.com>
 +
 +      parallel-tests: allow each test to have multiple results
 +      With this change, we improve the code creating the `test-suite.log'
 +      global log and the console testsuite summary to make it able to
 +      grasp multiple results per test script.  This is required in order
 +      to introduce the planned support for test protocols, like TAP and
 +      SubUnit, which can indeed run multiple tests per test script, each
 +      with its individual result.
 +      The implementation makes use of a custom reStructuredText field
 +      `:test-result:'.
 +      Note that no new documentation is added by this change; that is
 +      be left for follow-up changes.
 +      * lib/check.am ($(TEST_SUITE_LOG)): When processing .log files,
 +      recognize a report of a test's result only if it is declared with
 +      the custom `:test-result:' reStructuredText field placed at the
 +      beginning of a line.  Extend and add explanatory comments.
 +      (recheck, recheck-html): Add explanatory comments.
 +      * lib/test-driver: Write an appropriate reStructuredText field
 +      `:test-result:' in the generated log file.  Use a reStructuredText
 +      transition to better separate the test outcome report from the
 +      test script's registered output.  Improve comments.
 +      * tests/test-driver-custom-xfail-tests.test: Adjust.
 +      * tests/parallel-tests7.test: Adjust.
 +      * tests/parallel-tests-empty-testlogs.test: New test.
 +      * tests/parallel-tests-recheck-override.test: Likewise.
 +      * tests/parallel-tests2.test: Extend and keep more in-sync with ...
 +      * tests/test-driver-custom-html.test: ... this new related test.
 +      * tests/test-driver-custom-no-html.test: New test.
 +      * tests/test-driver-custom-multitest.test: Likewise.
 +      * tests/test-driver-custom-multitest-recheck.test: Likewise.
 +      * tests/test-driver-custom-multitest-recheck2.test: Likewise.
 +      * tests/trivial-test-driver: New file, used by the last four tests
 +      above.
 +      * tests/Makefile.am (TESTS): Update.
 +      (EXTRA_DIST): Distribute `trivial-test-driver'.
 +      (test-driver-custom-multitest.log): Depend on `trivial-test-driver'.
 +      (test-driver-custom-multitest-recheck.log): Likewise.
 +      (test-driver-custom-multitest-recheck2.log): Likewise.
 +      (test-driver-custom-html.log): Likewise.
 +
 +2011-06-21  Stefano Lattarini  <stefano.lattarini@gmail.com>
 +
 +      parallel-tests: allow custom driver scripts
 +      Allow suffix-based definition of custom "driver script" for the
 +      test scripts.  These driver scripts will be responsible of
 +      launching the tests (or their corresponding $(LOG_COMPILER), if
 +      they have an associated one), interpreting and displaying the
 +      test results, and writing the `.log' files.
 +      This new API should allow easy and flexible use of different
 +      test protocols in the future; in particular, we plan to use it
 +      to implement TAP and SubUnit harnesses.
 +      Note that no new documentation is added by this change; that is
 +      be left for follow-up changes.
 +      * automake.in (handle_tests): Define default for $(LOG_DRIVER),
 +      and, for any registered test extension `<ext>', define defaults
 +      for $(<ext>_LOG_DRIVER).  Substitute %DRIVER% using these new
 +      variables, instead of the old internal $(am__test_driver).  When
 +      processing check2.am, also substitute %DRIVER_FLAGS%.
 +      Require auxiliary script `test-driver' only if no driver has been
 +      explicitly defined for the test script kinds.
 +      * am/check2.am (?GENERIC?%EXT%$(EXEEXT).log, ?GENERIC?%EXT%.log,
 +      ?!GENERIC?%OBJ%): Pass the %DRIVER_FLAGS% to the %DRIVER% call.
 +      * tests/parallel-tests-no-extra-driver.test: New test.
 +      * tests/test-driver-custom.test: Likewise.
 +      * tests/test-driver-custom-xfail-tests.test: Likewise.
 +      * tests/test-driver-fail.test: Likewise.
 +      * tests/Makefile.am: Update.
 +      * NEWS: Update.
 +
 +2011-06-21  Stefano Lattarini  <stefano.lattarini@gmail.com>
 +
 +      parallel-tests: add auxiliary script 'test-driver', refactor
 +      This refactoring should cause no API of functionality change,
 +      and is meant only to simplify the future implementation of TAP
 +      and SubUnit testsuite drivers.  More precisely, our roadmap is
 +      to move most of the "testsuite driving" features out of the
 +      Automake-generated Makefiles, and into external scripts with
 +      well-defined interfaces.  This will allow the user to define
 +      its own personalized testsuite drivers, and will also offer us
 +      a framework upon which to implement our new TAP and SubUnit
 +      drivers, all in a very unobtrusive way and retaining an high
 +      degree of code reuse and backward-compatibility.
 +      * lib/test-driver: New auxiliary script.
 +      * lib/Makefile.am (dist_SCRIPT_DATA): Add it.
 +      * automake.in (handle_tests): Require the new auxiliary script
 +      `test-driver', and define a new internal makefile variable
 +      `$(am__test_driver)', used to call it.  Perform new substitution
 +      on `DRIVER' when processing the `check2.am' file.
 +      * lib/check.am (am__tty_colors): Define new shell variable
 +      `$am__color_tests'.
 +      (am__rst_section): Removed, its role taken over by the new
 +      `test-driver' script.
 +      (am__test_driver_flags): New variable, contains the command
 +      line options passed to `test-driver'.
 +      (am__check_pre): Do not deal with temporary files and exit
 +      traps anymore, as the `test-driver' script takes care of that
 +      now.  Define shell variable `$am__enable_hard_errors', used by
 +      `$(am__test_driver_flags)'.  Reorder so that we don't need to
 +      save and restore the value of the `TERM' environment variable
 +      anymore.
 +      Other related adjustments.
 +      (am__check_post): Remove, as its role has been completely taken
 +      over by the `test-driver' script.
 +      * am/check2.am (?GENERIC?%EXT%$(EXEEXT).log, ?GENERIC?%EXT%.log,
 +      ?!GENERIC?%OBJ%): Call the test script through the Automake
 +      substituted `%DRIVER%', and honor the command-line options
 +      in `$(am__test_driver_flags)'.  Do not call the obsoleted
 +      `$(am__check_post)' anymore.
 +      * doc/automake.texi (Auxiliary Programs): Mention the new
 +      `test-driver' script.
 +      (Optional): Mention `test-driver' in AC_CONFIG_AUX_DIR.
 +      Since we are at it, break the list of auxiliary scripts by
 +      placing one per line, to simplify potential future additions
 +      of new scripts.
 +      * tests/check.test: Adjust.
 +      * tests/check2.test : Likewise.
 +      * tests/check3.test : Likewise.
 +      * tests/check4.test : Likewise.
 +      * tests/check10.test: Likewise.
 +      * tests/color.test: Likewise.
 +      * tests/color2.test: Likewise.
 +      * tests/comment9.test: Likewise.
 +      * tests/dejagnu.test: Likewise.
 +      * tests/exeext4.test: Likewise.
 +      * tests/maken3.test: Likewise.
 +      * tests/maken4.test: Likewise.
 +      * tests/parallel-tests-interrupt.test: Likewise.
 +      * tests/posixsubst-tests.test: Likewise.
 +      * tests/repeated-options.test: Likewise.
 +      * tests/check-no-test-driver.test: New test.
 +      * tests/parallel-test-driver-install.test: Likewise.
 +      * tests/Makefile.am (TESTS): Update.
 +      * NEWS: Update.
 +
  2011-06-21  Stefano Lattarini  <stefano.lattarini@gmail.com>
  
        maintcheck: extend 'sc_tests_plain_*' checks
diff --cc automake.in
Simple merge
Simple merge
index f1ccb7b216a1f5a8cfc20a50e32dfe374d37ea30,3883dc009d6962a63bd1227f9a009dd3c02f1d21..b49e754e60fa93b95aec347f42c3000fd720f0e8
@@@ -723,17 -723,10 +724,21 @@@ parallel-tests-unreadable-log.test 
  parallel-tests-subdir.test \
  parallel-tests-interrupt.test \
  parallel-tests-reset-term.test \
+ parallel-tests-harderror.test \
+ parallel-tests-log-override-1.test \
+ parallel-tests-log-override-2.test \
+ parallel-tests-log-override-recheck.test \
 +parallel-tests-empty-testlogs.test \
 +parallel-test-driver-install.test \
 +test-driver-custom-no-extra-driver.test \
 +test-driver-custom.test \
 +test-driver-custom-xfail-tests.test \
 +test-driver-custom-multitest.test \
 +test-driver-custom-multitest-recheck.test \
 +test-driver-custom-multitest-recheck2.test \
 +test-driver-custom-html.test \
 +test-driver-custom-no-html.test \
 +test-driver-fail.test \
  parse.test \
  percent.test \
  percent2.test \
index f06562b458e9119bea5712b3ec85c113d23108ef,74538637b67fb7d8b062ef5498f89f2a110b7a45..0219eea21af49145c9029df85d0a9b3f1ff671cf
@@@ -983,17 -996,10 +984,21 @@@ parallel-tests-unreadable-log.test 
  parallel-tests-subdir.test \
  parallel-tests-interrupt.test \
  parallel-tests-reset-term.test \
+ parallel-tests-harderror.test \
+ parallel-tests-log-override-1.test \
+ parallel-tests-log-override-2.test \
+ parallel-tests-log-override-recheck.test \
 +parallel-tests-empty-testlogs.test \
 +parallel-test-driver-install.test \
 +test-driver-custom-no-extra-driver.test \
 +test-driver-custom.test \
 +test-driver-custom-xfail-tests.test \
 +test-driver-custom-multitest.test \
 +test-driver-custom-multitest-recheck.test \
 +test-driver-custom-multitest-recheck2.test \
 +test-driver-custom-html.test \
 +test-driver-custom-no-html.test \
 +test-driver-fail.test \
  parse.test \
  percent.test \
  percent2.test \