2011-06-20 Stefano Lattarini <stefano.lattarini@gmail.com>
+ maintcheck: avoid few more spurious failures
+ * tests/depmod-data.test: Use creative quoting to avoid
+ spuriously triggering the `sc_tests_Exit_not_exit' maintainer
+ check. Commit `v1.11-900-g3453b8e' attempted to fix it, but
+ succeeded only partially.
+ * tests/cond33.test: Fix header comments, not to reference
+ obsolescent make variable `$(mkdir_p)'.
+ * tests/cond4.test: Rewrite "$MAKE exp=..." as "exp=.. $MAKE -e",
+ to please `sc_tests_overriding_macros_on_cmdline'
+ * tests/cond19.test: Likewise.
+ * tests/cond32.test: Likewise.
+ * tests/add-missing.test: Use "AUTOMAKE_fails" instead of
+ "$AUTOMAKE ... && Exit 1", to please `sc_tests_automake_fails'.
+
++2011-06-20 Stefano Lattarini <stefano.lattarini@gmail.com>
++
+ maintcheck: avoid few spurious failures
+ * Makefile.am (sc_tests_plain_aclocal, sc_tests_plain_perl,
+ sc_tests_plain_autoconf, sc_tests_plain_automake,
+ sc_tests_plain_autoupate): Be stricter in matching an erroneous
+ literal command, i.e., `aclocal', `automake', `perl', etc.
+
2011-06-20 Bert Wesarg <bert.wesarg@googlemail.com> (tiny change)
check: don't use multi-line coloring for the report
See also following CC:ed thread on bug-autoconf list:
<http://lists.gnu.org/archive/html/bug-autoconf/2011-06/msg00002.html>
+2011-06-16 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: make test 'self-check-reexec.test' more portable
+ * tests/self-check-reexec.test: Rewrite not to require a Korn
+ Shell able to grok variable expansions such as `${.sh.version}';
+ Solaris 10 /bin/ksh fails on this for example. Instead, just
+ require bash and a shell that is not bash.
+
+2011-06-14 Stefano Lattarini <stefano.lattarini@gmail.com>
+ Jim Meyering <meyering@redhat.com>
+
+ test defs: fix ksh-related portability bug in warning messages
+ Running "make check" normally prints a diagnostic to the outermost
+ stderr (usually a tty) to explain why a test is skipped, thus
+ giving better and faster feedback to the user. It used to do
+ so by redirecting file descriptor 9 to stderr (via "exec 9>&2")
+ before invoking the test scripts, which then would write any skip
+ explanation to file descriptor 9 via the `skip_' function defined
+ in `tests/defs'.
+ However, various Korn Shells (at least Solaris 10's /bin/ksh and
+ Debian GNU/Linux's /bin/ksh) and the HP-UX's /bin/sh close open
+ file descriptors > 2 upon an `exec' system call; thus the effects
+ of "exec 9>&2" are cancelled upon fork-and-exec, so we would get
+ a "Bad file number" diagnostic and no skip explanation with those
+ shells.
+ The present change remedies this situation.
+ * tests/Makefile.am (AM_TESTS_ENVIRONMENT): Redirect more portably,
+ via a trailing "9>&2", rather than the prior "exec 9>&2; ...". Add
+ explanatory comments.
+ * tests/defs (stderr_fileno_): Update the advice in comments.
+ Based on commit v8.12-82-g6b68745 "tests: accommodate HP-UX and
+ ksh-derived shells" in GNU coreutils.
+ Further references, with lots of discussion:
+ <http://lists.gnu.org/archive/html/bug-autoconf/2011-06/msg00002.html>
+ <http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/22488>
+ <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=8846>
+
+2011-06-13 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: remove 'test_prefer_config_shell' from the environment
+ Since commit `v1.11-910-g7df1a9b', the once user-overridable
+ variable `$test_prefer_config_shell' has become an internal
+ detail, and the test scripts now complain and bail out if it is
+ set in the environment.
+ * tests/Makefile.am (AM_TESTS_ENVIRONMENT): Unset the variable
+ `test_prefer_config_shell' if it is set in the environment.
+
+2011-06-13 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: autogenerate list of wrapped tests for `lib/' shell scripts
+ * tests/gen-config-shell-tests: New script, generates distributed
+ makefile snippet `tests/config-shell-tests.am' to list all tests
+ that use the `get_shell_script' function, with names mangled to
+ use suffix `-w.shtst', in ...
+ * tests/Makefile.am (config_shell_tests): ... this macro, whose
+ definition has been consequently removed from Makefile.am.
+ (EXTRA_DIST): Distribute the new script.
+ ($(srcdir)/config-shell-tests.am): Generate using the new script.
+ (include): Include the `config-shell-tests.am' fragment.
+ * bootstrap: Invoke `tests/gen-config-shell-tests' to generate
+ `tests/config-shell-tests.am'.
+ * tests/.gitignore: Ignore `config-shell-tests.am'.
+ * tests/gen-parallel-tests: Fixlet in heading comments.
+
+2011-06-13 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: test mdate-sh with /bin/sh too
+ * tests/mdate5.test: Fetch the `mdate-sh' script using the
+ `get_shell_script' function, and run it directly instead of
+ using `$SHELL'.
+ * tests/mdate6.test: Likewise. Since we are at it, make checks
+ on the `mdate-sh' output stricter, remove now unneeded calls to
+ aclocal and automake and creation/extension of `configure.in',
+ `Makefile.am' and `textutils.tex' files, and add a trailing `:'
+ command.
+ * tests/Makefile.am (config_shell_tests): Add `mdate5-w.shtst'
+ and `mdate6-w.shtst'.
+
+2011-06-13 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: extend tests on `--add-missing' and `--copy' a bit
+ * tests/add-missing.test: Fix typo in heading comments. Try with
+ another testcase that install many (but not all) the auxiliary
+ scripts at once, and uses non-standard (but valid and documented)
+ setups (e.g., defining YACC in Makefile.am instead of calling
+ AC_PROG_YACC from configure.in).
+ * tests/copy.test: Reference `add-missing.test' in heading
+ comments. Try few more test scenarios.
+
+2011-06-16 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: few fixlets and improvements
+ * tests/cond31.test ($required): Remove `cc', it's not really
+ needed.
+ * tests/confh.test: Call autoheader too. The lack of this call
+ wasn't causing spurious failures because the automatic remake
+ rules were somehow invoking it on our behalf (at make time).
+ * tests/fn99subdir.test: Use $subdirname throughout, instead of
+ ${subdirname}, for consistency with the rest of the testsuite.
+ Avoid an unnecessary subshell, which could also cause spurious
+ passes, being guarded by a trailing `|| Exit 1', which neutralize
+ the `errexit' flag. Remove an unnecessary `|| Exit 1' guard.
+ * tests/insh2.test: Rewrite to avoid hackish Makefile.in munging,
+ and to also run configure and make.
+
+2011-06-13 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: don't hard-code test name in txinfo21.test
+ * tests/txinfo21.test: Use `$me' instead of hard-coding the
+ current testcase name "txinfo21". Add a trailing `:' command
+ since we are at it.
+
+2011-06-10 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: new test dedicated to `--add-missing' and `--copy'
+ * tests/add-missing.test: New test.
+ * tests/Makefile.am (TESTS): Update.
+ Suggested by Peter Rosin.
+
+2011-06-08 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: use 'fatal_' and 'framework_failure_' for hard errors
+ * tests/defs (require_xsi): Use `fatal_', not `framework_failure',
+ to report an invalid usage.
+ * tests/remake-gnulib-remove-header.test: Prefer using `fatal_'
+ with a proper error message over a direct call to `Exit 99'.
+ * tests/pr8365-remake-timing.test: Likewise.
+ * tests/cygnus-imply-foreign.test: Likewise.
+ * tests/missing6.test: Likewise.
+ * tests/cond8.test: Likewise.
+ * tests/cond33.test: Likewise.
+ * tests/python-virtualenv.test: Prefer using `framework_failure_'
+ with a proper error message over a direct call to `Exit 99'.
+ * tests/instspc-tests.sh: Prefer using `framework_failure_' and
+ `fatal_' over direct calls to `Exit 99'.
+ (fatal_): Define this (which is a simplified version of the one
+ in `tests/defs') for early uses (i.e., before `tests/defs'
+ gets sourced).
+ * tests/depmode-tests.sh: Likewise. Also, simplify the
+ 'get_depmodes' function and calls to it accordingly.
+
+2011-06-08 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ self tests: check new 'fatal_' function
+ * tests/self-check-exit.test: Also check the new 'fatal_'
+ function.
+
2011-06-08 Stefano Lattarini <stefano.lattarini@gmail.com>
test defs: new function 'fatal_', for hard errors
other reasons. The best way to fix this inconsistency is to
introduce a new function with a more general error message.
Inspired by a recent similar change to Gnulib's tests/init.sh.
- * tests/defs.in (fatal_): New function.
+ * tests/defs (fatal_): New function.
* tests/README (Section "Writing test cases" subsection "Do"):
Suggest the use of `fatal_', not of `framework_failure_', for
generic hard errors. The latter should be reserved for "real"
set-up failures.
+2011-06-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: `lib/' shell scripts transparently tested also with $SHELL
+ With the previous commit, the user could prefer the use of $SHELL
+ over /bin/sh in some tests checking the Automake-provided shell
+ scripts by manually exporting `test_prefer_config_shell' to "yes"
+ in the environment. With this commit, we ensure that such tests
+ *always* and *transparently* run using both $SHELL and /bin/sh
+ to execute the checked scripts. The `test_prefer_config_shell'
+ variable becomes an internal detail, and is no more meant to be
+ manually defined or overridden.
+ * tests/defs-static.in: Check that `test_prefer_config_shell' is
+ not exported in the environment. Error out if this is the case.
+ * tests/config-shell-tests.sh: New file, driver script to run
+ checks on the shell scripts in `lib/' using the $SHELL determined
+ at configure time instead of the default system shell /bin/sh.
+ * Makefile.am (TESTS_EXTENSIONS): Add `.shtst'.
+ (SHTST_LOG_COMPILER): Define, it calls `config-shell-tests.sh'.
+ (config_shell_tests): Define to a list of tests that wraps other
+ `*.test' tests using `config-shell-tests.sh'.
+ ($(config_shell_tests)): Dummy dependency declaration required
+ in order to have make actually produce expected log files from
+ the `.shtst.log' suffix rule.
+ (EXTRA_DIST): Distribute `config-shell-tests.sh'.
+ (TESTS): Add `$(config_shell_tests)'.
+ * tests/self-check-env-sanitize.test: Update, by checking that
+ `test_prefer_config_shell' isn't exported in the environment.
+
+2011-06-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: can use also $SHELL to check shell scripts from `lib/'
+ * tests/ar-lib.test: If the variable `$test_prefer_config_shell'
+ is set to "yes", run the script under test with configure-time
+ determined $SHELL, rather than with /bin/sh.
+ The `$test_prefer_config_shell' variable defaults to empty, but
+ can be overridden at runtime by the user, thus allowing more
+ coverage.
+ * tests/compile.test: Likewise.
+ * tests/compile2.test: Likewise.
+ * tests/compile3.test: Likewise.
+ * tests/compile4.test: Likewise.
+ * tests/compile5.test: Likewise.
+ * tests/compile6.test: Likewise.
+ * tests/instsh2.test: Likewise.
+ * tests/instsh3.test: Likewise.
+ * tests/mkinst3.test: Likewise.
+ * tests/missing.test: Likewise.
+ * tests/missing2.test: Likewise.
+ * tests/missing3.test: Likewise.
+ * tests/missing5.test: Likewise.
+ * tests/defs (get_shell_script): New subroutine, factoring out
+ code common to the tests above.
+ (xsi-lib-shell): If `$test_prefer_config_shell' is set to "yes",
+ check that $SHELL, not /bin/sh, supports XSI constructs, as we
+ expect the test will use $SHELL and not /bin/sh to run the
+ script being tested.
+
+2011-06-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests defs: better requirements for XSI shells
+ This change avoids potential spurious failures with tests using
+ the requirement 'xsi-shell' to mean that they want */bin/sh* (not
+ $SHELL) to be XSI-conforming. This idiom used to work before
+ commit `v1.11-874-g1321be7' (as back then the test scripts were
+ unconditionally run with /bin/sh), but has become inconsistent
+ now that the test scripts re-execute themselves with configure
+ determined $SHELL.
+ The described spurious failures have already occurred in practice,
+ for examples on Solaris systems which also had GNU Bash installed.
+ From a suggestion by Peter Rosin. See discussion at:
+ <http://lists.gnu.org/archive/html/automake-patches/2011-06/msg00016.html>
+ * tests/defs (xsi-shell): Now check that $SHELL, rather than the
+ shell currently running the test script, is an XSI shell.
+ (xsi-bin-sh): New requirement, checking that /bin/sh (which can
+ differ from $SHELL) is an XSI shell.
+ (xsi-lib-shell): New requirement, checking that the shell that
+ should be used to test the Automake-provided scripts from `lib/'
+ is an XSI shell. For the moment, this is just an alias for
+ `xsi-bin-sh'.
+ (require_xsi): New subroutine, used to factor out code common to
+ the requirements above.
+ ($xsi_shell_code): New variable, contains shell code supposed to
+ work only with XSI shells. Used by the new subroutine above.
+ * tests/ar-lib.test ($required): Require 'xsi-lib-shell' instead
+ of 'xsi-shell', since the script we test here is run with /bin/sh,
+ not with $SHELL.
+ * tests/compile3.test: Likewise.
+ * tests/compile6.test: Likewise.
+
2011-06-05 Stefano Lattarini <stefano.lattarini@gmail.com>
tests: fix typo-related error in auxdir2.test
its failure went unnoticed, even if it was due to a wrong cause.
Bug introduced in commit v1.11-249-g49ac3de.
+2011-06-05 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ self tests: check that `$me' can be overridden
+ * tests/self-check-me.test: Check that `$me' can be overridden
+ before sourcing ./defs, with or without sourcing ./defs-static
+ beforehand, and that this override is honored. Update heading
+ comments.
+
+2011-06-02 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ self tests: fix another spurious failure
+ Our ad-hoc usage of `tests/defs' in the testsuite's self tests
+ stopped working properly when we made the test scripts re-execute
+ themselves with the configure-time $SHELL. Fix this.
+ Fixes a regression introduced by commit 'v1.11-874-g1321be7'.
+ * tests/self-check-exit.test: Export `AM_TESTS_REEXEC' to "no"
+ before running the self tests. This fixes a spurious failure
+ present only when the test was run by hand.
+
+2011-06-02 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ silent-rules tests: fix spurious failures with Sun Studio C++
+ * tests/silentcxx.test: The C++ compiler from Sun Studio is named
+ `CC'. Account for this in our grepping checks on the make output.
+ Since we are at it, throw in a couple of improvements to comments
+ and formatting.
+ * tests/silent-many-generic.test: The C++ compiler from Sun Studio
+ is named `CC', and this can cause spurious failures in our grepping
+ of the make output. Work around this by using a wrapper script
+ around the C++ compiler (generated on the fly), since filtering the
+ make output proved to be too fragile.
+
+2011-06-02 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: fix spurious failure in backcompat2.test on NetBSD
+ * tests/backcompat2.test: Add trailing `:' in the body of a `for'
+ loop, in case the last command there might have an exit status
+ different 0. This work around `set -e' issues in some BSD shells,
+ e.g., NetBSD /bin/ksh.
+
+2011-06-02 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: fix spurious failure in autohdr3.test
+ * tests/autohder3.test (Makefile.am): Let `test' depend on `all',
+ so that config.h header is truly remade.
+ Remove now useless call to `$MAKE' when non-GNU make is in use.
+ Failure introduced by commit `v1.11-895-g5e62b96'.
+
+2011-06-02 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: fix spurious failure of cond29.test on NetBSD
+ * tests/cond29.test: Skip if we cannot safely limit the maximal
+ size of used virtual memory to 20K. This fixes a spurious failure
+ on NetBSD.
+ Bug introduced in commit `v1.11-885-g908d335'.
+
+2011-06-02 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ maintcheck: fix again few more failures
+ * tests/cond29.test: To please the `sc_tests_Exit_not_exit'
+ maintainer check, avoid using `sh -c "exit 0"' where a simple
+ `sh -c :' will do.
+ * tests/depmod-data.test: Use creative quoting to avoid
+ spuriously triggering the `sc_tests_Exit_not_exit' maintainer
+ check.
+
2011-06-02 Stefano Lattarini <stefano.lattarini@gmail.com>
maintcheck: fix some more failures
check.
* tests/depcomp8b.test: Likewise.
+2011-06-02 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: don't require GNU make where it's not strictly needed
+ * tests/autohdr3.test: Rewrite to be stricter when make is GNU
+ make. Drop the requirement of GNU make.
+ * tests/aclocal5.test: Drop GNU make requirement, it's not truly
+ needed. Add reference to similar tests `remake-subdir*.test'.
+ * remake-subdir-gnu.test, remake-subdir-from-subdir.test,
+ remake-subdir.test, remake-subdir2.test: Add reference to
+ each other, and to related test `aclocal5.test'.
+ * tests/aclocal6.test: Drop GNU make requirement, it's not
+ truly needed.
+ * tests/confh6.test: Likewise.
+ * tests/lex3.test: Likewise.
+ * tests/remake11.test: Likewise.
+ * tests/subdir5.test: Likewise.
+ * tests/subdir8.test: Likewise.
+ * tests/werror2.test: Likewise.
+ * tests/conff.test: Likewise, and ensure verbose printing of
+ captured make output.
+ * tests/lex5.test: Tweak so that GNU make is no more required.
+ * tests/version7.test: Likewise.
+ * tests/maken2.test: Add explicative comment for why this test
+ requires GNU make.
+ * tests/maken4.test: Let it run also with BSD makes supporting
+ the `.MAKE' special target.
+ * tests/output6.test: Use proper m4 quoting in configure.in.
+ Expand make macros with one-character name using `$(x)', not
+ `$x', for portability. Move checks in the makefiles, rather
+ than relying on grepping the output from make. Drop the now
+ unneeded GNU make requirement.
+
2011-05-29 Stefano Lattarini <stefano.lattarini@gmail.com>
remake: behave better with non-GNU make in subdirectories
* tests/remake-subdir-from-subdir.test: Likewise.
* tests/Makefile.am (TESTS): Update.
+2011-06-01 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ lex tests: fix spurious failures with Solaris lex
+ * tests/lex-lib.test (foo.l): Avoid empty "rules section", which
+ can confuse Solaris lex.
+ * tests/lex-libobj.test (foo.l): Likewise.
+
+2011-06-01 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ lex tests: do not force the use of flex unconditionally
+ * tests/defs (lex): Act more similarly to what the `yacc'
+ requirement does, i.e., only force the use of flex if the
+ $LEX variable is left unset by the user.
+ (flex): Use `skip_' to skip the test if flex is not found.
+
+2011-06-01 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ lex tests: avoid possible hang; fix and extend
+ * tests/lex3.test (foo.l:yywrap): Return 1, not 0, to avoid hangs.
+ Bug introduced in commit 'v1.11-871-geb147a1'.
+ (Makefile.am): Do not add `@LEXLIB@' to `$(LDADD)', as we define
+ our own `yywrap' function.
+ * tests/lex.test (tscan.l): In `yywrap', return 1, not 0, for
+ consistency with the default flex implementation.
+ * tests/lex-libobj.test (yywrap.c): Likewise.
+ * tests/lex-subobj-nodep.test (s1.l): Likewise.
+ * tests/lexvpath.test (foo.c): Likewise.
+ * tests/silent-lex-gcc (foo.l): Likewise.
+ * tests/silent-lex-generic (foo.l): Likewise.
+ * tests/silent-many-gcc (foo5.l): Likewise.
+ * tests/silent-many-generic (foo5.l): Likewise.
+ * tests/lex-lib.test (mu.c): Likewise.
+ Update heading comments, to refer to ...
+ * tests/lex-lib-external.test: ... this new test, which checks
+ that we can get use the `yywrap' function from a system-wide
+ library, if that's available.
+
+2011-06-01 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: prefer `skip_' over `echo ...; Exit 77'
+ * tests/self-check-cleanup.test: When the test must be skipped,
+ use `skip_ REASON' instead of `echo REASON; Exit 77'. Also,
+ make the skip message shorter and clearer.
+
+2011-06-01 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: fix spurious failures in self tests
+ Our ad-hoc usage of `tests/defs' in the testsuite's self tests
+ stopped working properly when we made the test scripts re-execute
+ themselves with the configure-time $SHELL. Fix this.
+ Fixes a bug introduced by commit 'v1.11-874-g1321be7'.
+ * tests/defs: Only check that we can find the client test script
+ when we must re-execute it.
+ * tests/self-check-cleanup.test: Export `AM_TESTS_REEXEC' to "no"
+ before running the self tests.
+ * tests/self-check-dir.test: Likewise.
+ * tests/self-check-explicit-skips.test: Likewise.
+ * tests/self-check-me.test: Likewise.
+ * tests/self-check-sanity.test: Likewise.
+ * tests/self-check-reexec.test: New test.
+ * tests/Makefile.am (TESTS): Update.
+
+2011-06-01 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: prefer ulimit over timeout in test on a memory-hogging bug
+ * tests/cond29.test: Use 'ulimit' instead of 'timeout' to ensure
+ that automake does not uses up too much resources. This is really
+ and improvement because the bug tested by this script wasn't just
+ a "it hangs" or "it runs too slow" bug, but rather a memory-hogging
+ bug (due to combinatorial explosion when many Automake conditionals
+ had to be handled) which could easily crash the whole system, which
+ is unacceptable. The requirement of a proper and working 'ulimit'
+ builtin might cause the test to be skipped on more systems, but
+ that shouldn't be a problem since the bug isn't about a portability
+ issue, but is rather an automake internal implementation problem.
+ I've verified that the test as updated by this patch still passes
+ with automake 1.8.5, automake 1.10.2, and obviously the development
+ version of automake, and that it fails with automake 1.7.9.
+
+2011-06-01 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: avoid generating `*-p.test' tests, use a wrapper script
+ The generated `*-p.test' tests had already become just thin layers
+ around the corresponding test scripts. This change makes the final
+ step, converting to the use of a generic wrapper script and thus
+ avoiding the extra test generation (similarly to what is done for
+ the `*.instspc' and `*.depmod' tests).
+ * tests/parallel-tests.sh: New file, driver script to run checks
+ the on the `parallel-tests' semantics by wrapping tests that use
+ the generic "Simple Tests" driver.
+ * tests/gen-parallel-tests: Update, mostly to reflect the new
+ `.ptest' extensions used for tests in $(parallel_tests).
+ * Makefile.am (TESTS_EXTENSIONS): Add `.ptest'.
+ (PTEST_LOG_COMPILER): Define, it calls `parallel-tests.sh'.
+ ($(parallel_tests)): Do not really generate `*-p.test' tests
+ anymore; this is now just a dummy dependency declaration required
+ in order to have make actually produce expected log files from
+ the `.ptest.log' suffix rule.
+ (EXTRA_DIST): Distribute `parallel-tests.sh'.
+ (MAINTAINERCLEANFILES): Don't remove the `$(parallel_tests)', it
+ is not necessary anymore.
+ (generated_tests): Variable definition removed.
+ (TESTS): Update, by listing `$(parallel_tests)' directly instead
+ of `$(generated_tests)'
+ (expected_list_of_tests): Remove `$(generated_tests)'.
+ (maintainer-check-list-of-tests): No need to explicitly depend on
+ `$(expected_list_of_tests)' anymore.
+
2011-05-29 Stefano Lattarini <stefano.lattarini@gmail.com>
automake, aclocal: honour configure-time AUTOCONF and AUTOM4TE
the `DESTDIR' variable, is poorly chosen, if not downright wrong.
Rewrite it to use `prefix' as the overridden variable instead.
+2011-05-29 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: don't require gfortran if any fortran compiler is enough
+ * tests/silentf90.test ($required): Require `fortran', not
+ `gfortran'.
+ * tests/silentf77.test ($required): Require `fortran77', not
+ `gfortran'.
+ * tests/silent-many-generic.test ($required): Require `fortran'
+ and `fortran77' rather than `gfortran'.
+
+2011-05-27 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: improve `ccnoco*.test', better cross-compiling support
+ * tests/ccnoco3.test (Mycomp): Use the `$CC' chosen by `tests/defs'
+ instead of forcing `gcc' unconditionally. This ensures better
+ coverage in case of cross-compiling, when GCC can be named e.g.,
+ `i586-mingw32msvc-gcc', instead of simply `gcc'.
+ * tests/ccnoco.test: Likewise. Remove redundant checks. Modernize
+ the created `configure.in'. Run tests both in-tree and in VPATH.
+ Export `CC' to the overridden value only once.
+ * tests/ccnoco2.test: Slightly stricter grepping of automake
+ stderr. Add trailing `:' command.
+
2011-05-27 Stefano Lattarini <stefano.lattarini@gmail.com>
testsuite: each test case depends on `defs-static'
* tests/Makefile.am ($(TEST_LOGS)): Depends on `defs-static' too.
Simplify comments.
+2011-05-27 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: use $SHELL to run tests which are shell scripts
+ * tests/Makefile.am (TEST_LOG_COMPILER): Define so that the
+ configure-time $SHELL is used to run the tests.
+ * tests/defs: Add code to re-execute by default the running
+ test script with configure-time $SHELL. Updated comments.
+ * configure.ac: Check that `set -e' is working for $SHELL,
+ not for /bin/sh.
+ * tests/defs-static.in: Update comments.
+ * tests/README (Supported shells): Updated.
+ (Getting details from failures): Don't tell that tests are
+ run by /bin/sh by default.
+
+2011-05-26 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: split 'subst2.test' to improve modularity and coverage
+ * tests/subst2.test: Test removed, split into ...
+ * tests/programs-primary-rewritten.test: ... this test ...
+ * tests/subst-no-trailing-empty-line.test: ... and this one ...
+ * tests/extra-programs-empty.test: ... and this one.
+ * tests/Makefile.am (TESTS): Update.
+
+2011-05-26 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: tweak and improve tests on Automake conditionals
+ The "avoid the requirement of a working compiler" we refer about
+ below is obtained by disabling automatic dependency tracking and
+ defining `CC' (and `EXEEXT', `OBJEXT', `CXX', etc., if needed) to
+ dummy values, either directly in the `Makefile.am', or by AC_SUBST
+ in `configure.in'.
+ The "cosmetic changes" we refer about below consists in, e.g.,
+ adding a trailing `:' command to a script, using proper m4 quoting
+ in `configure.in' files, adding commands that offer more debugging
+ output, explicitly declaring phony targets as such in Makefiles,
+ making use of idioms more consistent with those employed in other
+ tests, following the GNU formatting standards more closely, etc.
+ * tests/cond.test: Cosmetic changes.
+ * tests/cond3.test: Likewise.
+ * tests/cond10.test: Likewise.
+ * tests/cond14.test: Likewise.
+ * tests/cond15.test: Likewise.
+ * tests/cond17.test: Likewise.
+ * tests/cond40.test: Likewise.
+ * tests/cond13.test: Likewise, and make grepping of the generated
+ `Makefile.in' slightly stricter.
+ * tests/cond6.test: Extend by also running ./configure, make and
+ "make install". Few cosmetic changes.
+ * tests/cond8.test: Likewise.
+ * tests/cond2.test: Do few cosmetic changes. Make grepping of
+ automake stderr stricter.
+ * tests/cond20.test: Likewise.
+ * tests/cond23.test: Likewise.
+ * tests/cond24.test: Likewise.
+ * tests/cond42.test: Likewise.
+ * tests/cond46.test: Likewise.
+ * tests/cond9.test: Move more checks in the `Makefile.am' instead
+ of grepping make's output.
+ * tests/cond38.test: Likewise.
+ * tests/cond11.test: Likewise. Avoid the requirement of a working
+ C compiler.
+ * tests/cond16.test: Likewise.
+ * tests/cond22.test: Likewise.
+ * tests/cond30.test: Likewise.
+ * tests/cond31.test: Likewise.
+ * tests/cond4.test: Likewise, and drop the now-useless GNU make
+ requirement as well.
+ * tests/cond18.test: Likewise.
+ * tests/cond19.test: Likewise.
+ * tests/cond22.test: Likewise.
+ * tests/cond25.test: Add trailing `:' command.
+ * tests/cond26.test: Likewise.
+ * tests/cond27.test: Likewise.
+ * tests/cond28.test: Likewise.
+ * tests/cond29.test: Use `unindent' function for better formatting.
+ Other minor cosmetic changes. Use a `timeout' program (if it's
+ available) to determine whether the script takes too long, instead
+ of just hoping that the user will notice an abnormally long test
+ execution time.
+ * tests/cond33.test: Extend a bit. Fix heading comments. Few more
+ cosmetic changes.
+ * tests/cond37.test: Ensure verbose printing of captured make
+ output. Minor cosmetic changes.
+ * tests/cond34.test: Likewise. Avoid the requirement of a working
+ C compiler. Make grepping of make output slightly stricter.
+ * tests/cond35.test: Quote literal dots in grep regexp. Add extra
+ debugging output. Minor cosmetic changes.
+ * tests/cond36.test: Likewise.
+ * tests/cond41.test: Do not simply check that aclocal fails, but
+ also grep its stderr for the expected error message.
+ * tests/cond43.test: Likewise (but the program being automake).
+ Few cosmetic changes.
+ * tests/condd.test: Avoid the requirement of a working C compiler.
+ * tests/condman3.test: Extend by using more man pages and more
+ manpage sections.
+ * tests/condman.test: Removed, it's completely superseded by
+ `condman3.test'.
+ * tests/Makefile.am (TESTS): Update.
+
+2011-05-25 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ lex tests: avoid spurious failures when LEXLIB isn't found
+ The AC_PROG_LEX Autoconf macro does not diagnose a failure to find
+ the "lex library" expected to provide a `yywrap' function (function
+ which is required to link most lex-generated programs). On the
+ contrary, when all the link attempts (i.e., with `-ll' and `-lfl')
+ fail, configure declares that no lex library is needed, and simply
+ proceeds with the configuration process -- only for the build to
+ possibly fail later, at make time.
+ This behaviour might be (partly) intended; the Autoconf manual
+ reads:
+ ``You are encouraged to use Flex in your sources, since it is
+ both more pleasant to use than plain Lex and the C source
+ it produces is portable. In order to ensure portability,
+ however, you must either provide a function `yywrap' or, if
+ you don't use it (e.g., your scanner has no `#include'-like
+ feature), simply include a `%noyywrap' statement in the
+ scanner's source.''
+ This AC_PROG_LEX behaviour is causing some spurious failures of the
+ Automake testsuite in environments which lack a proper library
+ providing `yywrap' (this happens for example in Linux->MinGW cross
+ compilations). But at this point is clear that a proper workaround
+ is to simply provide a fall-back implementation of `yywrap' in our
+ lexers.
+ * tests/cond35.test: Provide a dummy `yywrap' function.
+ * tests/lex3.test: Likewise.
+ * tests/lexvpath.test: Likewise.
+ * tests/silent-many-gcc.test: Likewise.
+ * tests/silent-many-generic.test: Likewise.
+ * tests/silent-lex-gcc.test: Likewise, and a dummy `main' too.
+ * tests/silent-lex-generic.test: Likewise.
+ * tests/lex-lib.test: New test.
+ * tests/lex-libobj.test: New test.
+ * tests/lex-nowrap.test: New test.
+ * tests/Makefile.am (TESTS): Update.
+ * THANKS: Update.
+ Thanks to Russ Allbery for the suggestion.
+
+2011-05-25 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: require C++ compiler explicitly in tests needing it
+ The list of the test scripts which needed a C++ compiler but
+ failed to require it explicitly has been found by running:
+ $ tests=`grep -lE 'CXX|\.c(c|\+\+|xx|pp)' *.test | tr '\n' ' '`
+ $ am_explicit_skips=yes CXX=false make check TESTS="$tests"
+ and looking for tests that reported FAIL instead of SKIP.
+ After the present change, all those previously failing tests
+ either pass or get correctly skipped.
+ * tests/subobj9.test: Require `c++', instead of explicitly
+ skipping on configure failure. Add excerpts and/or details from
+ the original bug report that prompted this tests to be written.
+ * tests/silentcxx-gcc.test: Require g++.
+ * tests/silentcxx.test: Require c++.
+ * tests/suffix3.test: Likewise.
+
+2011-05-25 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: don't require g++ where any C++ compiler is enough
+ * tests/specflg10.test ($required): Use `c++', not g++.
+ * tests/silent-many-generic.test: Likewise.
+
+2011-05-25 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: require C compiler explicitly in tests needing it
+ The list of the test scripts which needed a C compiler but
+ failed to require it explicitly has been found by running:
+ $ am_explicit_skips=yes CC=false make check
+ and looking for tests that reported FAIL instead of SKIP.
+ After the present change, all those previously failing tests
+ either pass or get correctly skipped.
+ * tests/aclocal4.test ($required): Add `cc'.
+ * tests/ansi.test: Likewise.
+ * tests/ansi10.test: Likewise.
+ * tests/ansi3.test: Likewise.
+ * tests/ansi3b.test: Likewise.
+ * tests/ansi5.test: Likewise.
+ * tests/ansi6.test: Likewise.
+ * tests/ansi7.test: Likewise.
+ * tests/ansi9.test: Likewise.
+ * tests/backcompat6.test: Likewise.
+ * tests/canon7.test: Likewise.
+ * tests/check5.test: Likewise.
+ * tests/check7.test: Likewise.
+ * tests/check8.test: Likewise.
+ * tests/cond4.test: Likewise.
+ * tests/cond11.test: Likewise.
+ * tests/cond16.test: Likewise.
+ * tests/cond18.test: Likewise.
+ * tests/cond19.test: Likewise.
+ * tests/cond22.test: Likewise.
+ * tests/cond30.test: Likewise.
+ * tests/cond31.test: Likewise.
+ * tests/cond32.test: Likewise.
+ * tests/cond34.test: Likewise.
+ * tests/cond35.test: Likewise.
+ * tests/cond39.test: Likewise.
+ * tests/condd.test: Likewise.
+ * tests/cygnus-dependency-tracking.test: Likewise.
+ * tests/depcomp2.test: Likewise.
+ * tests/depcomp6.test: Likewise.
+ * tests/depcomp7.test: Likewise.
+ * tests/depcomp8a.test: Likewise.
+ * tests/depcomp8b.test: Likewise.
+ * tests/depdist.test: Likewise.
+ * tests/depend2.test: Likewise.
+ * tests/depend5.test: Likewise.
+ * tests/distcleancheck.test: Likewise.
+ * tests/distname.test: Likewise.
+ * tests/exeext.test: Likewise.
+ * tests/exeext4.test: Likewise.
+ * tests/extradep.test: Likewise.
+ * tests/extradep2.test: Likewise.
+ * tests/gnits2.test: Likewise.
+ * tests/gnits3.test: Likewise.
+ * tests/instdir-ltlib.test: Likewise.
+ * tests/instdir-prog.test: Likewise.
+ * tests/instfail.test: Likewise.
+ * tests/instfail-libtool.test: Likewise.
+ * tests/lex3.test: Likewise.
+ * tests/lex5.test: Likewise.
+ * tests/lexvpath.test: Likewise.
+ * tests/lex-subobj-nodep.test: Likewise.
+ * tests/lflags.test: Likewise.
+ * tests/libobj-basic.test: Likewise.
+ * tests/libobj2.test: Likewise.
+ * tests/libobj7.test: Likewise.
+ * tests/libobj10.test: Likewise.
+ * tests/libobj16a.test: Likewise.
+ * tests/libobj16b.test: Likewise.
+ * tests/libobj17.test: Likewise.
+ * tests/libobj19.test: Likewise.
+ * tests/libtool2.test: Likewise.
+ * tests/libtool3.test: Likewise.
+ * tests/libtool7.test: Likewise.
+ * tests/libtool9.test: Likewise.
+ * tests/libtoo10.test: Likewise.
+ * tests/libtoo11.test: Likewise.
+ * tests/ltcond.test: Likewise.
+ * tests/ltcond2.test: Likewise.
+ * tests/ltconv.test: Likewise.
+ * tests/ltinit.test: Likewise.
+ * tests/ltlibsrc.test: Likewise.
+ * tests/ltorder.test: Likewise.
+ * tests/nobase.test: Likewise.
+ * tests/nobase-libtool.test: Likewise.
+ * tests/mmodely.test: Likewise.
+ * tests/parallel-tests5.test: Likewise.
+ * tests/parallel-tests-suffix-prog.test: Likewise.
+ * tests/parallel-tests-ext-driver-prog.test: Likewise.
+ * tests/posixsubst-ldadd.test: Likewise.
+ * tests/posixsubst-libraries.test: Likewise.
+ * tests/posixsubst-ltlibraries.test: Likewise.
+ * tests/posixsubst-programs.test: Likewise.
+ * tests/posixsubst-sources.test: Likewise.
+ * tests/pr87.test: Likewise.
+ * tests/pr204.test: Likewise.
+ * tests/pr224.test: Likewise.
+ * tests/pr300-lib.test: Likewise.
+ * tests/pr300-ltlib.test: Likewise.
+ * tests/pr300-prog.test: Likewise.
+ * tests/pr401.test: Likewise.
+ * tests/pr401b.test: Likewise.
+ * tests/pr401c.test: Likewise.
+ * tests/remake-gnulib-add-acsubst.test: Likewise.
+ * tests/remake-gnulib-add-header.test: Likewise.
+ * tests/regex.test: Likewise.
+ * tests/repeated-options.test: Likewise.
+ * tests/silent.test: Likewise.
+ * tests/silent3.test: Likewise.
+ * tests/silent9.test: Likewise.
+ * tests/silent-lex-generic.test: Likewise.
+ * tests/silent-many-generic.test: Likewise.
+ * tests/silent-yacc-generic.test: Likewise.
+ * tests/specflg7.test: Likewise.
+ * tests/specflg9.test: Likewise.
+ * tests/specflg10.test: Likewise.
+ * tests/stdinc.test: Likewise.
+ * tests/strip.test: Likewise.
+ * tests/strip2.test: Likewise.
+ * tests/strip3.test: Likewise.
+ * tests/subdirbuiltsources.test: Likewise.
+ * tests/subobj3.test: Likewise.
+ * tests/subobj6.test: Likewise.
+ * tests/subobj11a.test: Likewise.
+ * tests/subpkg.test: Likewise.
+ * tests/subst2.test: Likewise.
+ * tests/subst3.test: Likewise.
+ * tests/substref.test: Likewise.
+ * tests/substre2.test: Likewise.
+ * tests/suffix5.test: Likewise.
+ * tests/suffix8.test: Likewise.
+ * tests/suffix10.test: Likewise.
+ * tests/suffix11.test: Likewise.
+ * tests/suffix12.test: Likewise.
+ * tests/suffix13.test: Likewise.
+ * tests/target-cflags.test: Likewise.
+ * tests/transform.test: Likewise.
+ * tests/transform2.test: Likewise.
+ * tests/yacc-basic.test: Likewise.
+ * tests/yacc-d-basic.test: Likewise.
+ * tests/yacc-clean.test: Likewise.
+ * tests/yacc-dist-nobuild.test: Likewise.
+ * tests/yacc-nodist.test: Likewise.
+ * tests/yaccvpath.test: Likewise.
+ * tests/yacc-d-vpath.test: Likewise.
+ * tests/yacc4.test: Likewise.
+ * tests/yacc7.test: Likewise.
+ * tests/yacc8.test: Likewise.
+ * tests/yaccdry.test: Likewise.
+ * tests/yflags-cmdline-override.test: Likewise.
+ * tests/yflags-force-override.test: Likewise.
+ * tests/python-virtualenv.test: Likewise. Also, improve skip
+ messages.
+ * tests/subobj5.test ($required): Add `cc'.
+ (Makefile.am): Set `AUTOMAKE_OPTIONS' to `subdir-objects', and
+ add new checking rules `test-build' and `test-distdir'.
+ Extend the test by building and examining the distdir, the
+ program, and the object files.
+ * tests/postproc.test ($required): Add `cc'.
+ Avoid the explicit `|| Exit $?' after call to configure, which
+ is now either redundant (e.g., when `am_explicit_skips' is
+ unset), or counter-productive (e.g., when `am_explicit_skips'
+ is set to "yes").
+ * tests/pr243.test: Likewise.
+ * tests/pr266.test: Likewise.
+ * tests/pr220.test: Simplify so that it doesn't require a
+ C compiler anymore.
+ * tests/subdir5.test: Likewise.
+ * tests/subdir8.test: Likewise.
+ * tests/lflags.test: Likewise.
+ * tests/yflags.test: Likewise.
+ * tests/yflags-force-conditional.test: Likewise.
+ * tests/lflags2.test: Simplify so that it doesn't require a
+ C++ compiler anymore.
+ * tests/yflags2.test: Likewise.
+ * tests/autohdrdry.test (configure.in): Remove unneeded call
+ to `AC_PROG_CC'.
+ * tests/pr287.test: Likewise.
+ * tests/check6.test: Likewise.
+ * tests/cond21.test: Likewise, plus some cosmetic adjustments.
+ * tests/upc.test: Skip if configure fails with status `77'.
+ * tests/upc3.test: Likewise.
+ * tests/vala4.test: Likewise.
+ * tests/nostdinc.test: Likewise. Also, make grepping checks
+ on `Makefile.in'. Update heading comments.
+ * tests/compile5.test: When we must skip, skip explicitly,
+ and with a meaningful message.
+ * tests/instspc-tests.sh: When running in "test-build" or
+ "test-install" mode (as determined by the value of variable
+ `$instspc_action'), require `cc'. Adjust comments.
+ * tests/depmod-tests.sh: Likewise.
+
+2011-05-24 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: allow user to ask for stricter skip semantics
+ Given how the Automake testsuite is currently structured, if a
+ command in a test script fails with status `77', the global test
+ outcome is considered a SKIP, because the value of `77' for `$?'
+ is passed to the exit trap. Indeed, this happens in practice, as
+ an autoconf-generated configure script can exit with status `77'
+ if it fails to find, e.g., a required compiler.
+ While this behaviour is quite useful for avoiding spurious test
+ failures in the wild, it can also sometimes prevent the Automake
+ developers to easily see and declare the requirements of their
+ tests.
+ This change introduces a new variable `am_explicit_skips', meant to
+ be user-overridable, and which, when set to a "true" value (i.e.,
+ `yes' or `1'), require a direct call to `Exit 77' in order to make
+ the test outcome be considered a SKIP.
+ * tests/defs.in ($am__test_skipped): New variable, initialized
+ to `no'.
+ (Exit): Set `$am__test_skipped' to `yes' if passed an exit status
+ of 77.
+ (trap '...' 0): When `$am_explicit_skips' is set to a "true" value,
+ reset an exit status of `77' to `78' if $am__test_skipped is not
+ set to `yes'.
+ * tests/self-check-exit.test: Adjust: unset `am_explicit_skips'.
+ * tests/self-check-explicit-skips.test: New test.
+ * tests/Makefile.am (TESTS): Update.
+
+2011-05-24 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: user can force skipping of compiler-requiring tests
+ * tests/defs (cc, c++, fortran, fortran77): Skip the test if
+ the relevant compiler is disabled by having the corresponding
+ variable (CC, CXX, FC and F77, respectively) set to "false".
+ (yacc): For consistency, skip the test when the YACC variable
+ is set to "false", not when it's set to "no". Since we are at
+ it, fix the skip message to be shorter and more consistent.
+
+2011-05-23 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: tweak and improve tests on "Simple Tests" driver
+ * tests/check.test: Make grepping of generated Makefile.in
+ slightly stricter. Add trailing `:' command.
+ * tests/check2.test: Ensure verbose printing of captured make
+ output. Make grepping of generated Makefile.in and of make
+ output stricter. Add trailing `:' command.
+ * tests/check3.test: Likewise. Also, fix minor error in a
+ comment.
+ * tests/check4.test: Stricter grepping of make output. Use
+ `using_gmake' subroutine instead of copy&paste from defs.in
+ to determine whether $MAKE is GNU make. Add a trailing `:'
+ command.
+ * tests/check6.test: Add trailing `:' command.
+ * tests/check10.test: Prefer redirection `>' over `>>' when
+ creating new files. Ensure verbose printing of captured make
+ output.
+ * tests/check11.test: Likewise.
+ * tests/check7.test: Likewise. Make grepping of make output
+ stricter. Prefer "make distcheck" over a simple "make check"
+ in a VPATH build. Add trailing `:' command.
+ (a.c): Format following GNU coding standards.
+ (Makefile.am): Explicitly declare target `print-xfail-tests'
+ as PHONY. Distribute $(check_SCRIPTS), for distcheck.
+ * tests/parallel-tests.test: Prefer redirection `>' over `>>'
+ when creating new files.
+ * tests/parallel-tests2.test: Likewise.
+ * tests/parallel-tests9.test: Likewise.
+ * tests/parallel-tests8.test: Likewise. Make grepping of
+ automake stderr stricter.
+ * tests/check8.test: Likewise. Make formatting of created
+ *.c files more consistent with GNU Coding Standards.
+ * tests/check-subst-prog.test (configure.in): Break overly
+ long lines.
+ (pass-prog.c, xfail-prog.c): Format them more consistently
+ with GNU Coding Standards.
+ * tests/parallel-tests3.test: Use a shorter and clearer skip
+ message. Use the `unindent' subroutine to improve readability
+ of indented loop bodies. Simplify the hack used to please
+ maintainer-check.
+ * tests/parallel-tests5.test: Make formatting of created *.c
+ files more consistent with GNU Coding Standards. Use the
+ `unindent' subroutine to improve readability of indented loop
+ bodies.
+ * tests/parallel-tests10.test: Add blank line, for clarity.
+ Prefer trailing `:' over trailing `Exit 0', for consistency
+ with other tests.
+
+2011-05-21 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests/README: small consistency improvement
+ * tests/README (Section "User interface" subsection "Running the
+ tests"): In examples, be more consistent w.r.t. the differences
+ between GNU and non-GNU make.
+
+2011-05-21 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests/README: suggest how to run tests in cross-compile mode
+ * tests/README (Section "User interface" subsection "Running the
+ tests"): Briefly explain how to override 'host_alias' at runtime
+ to force the use of cross-compilers by the testsuite. Give an
+ example.
+
+2011-05-21 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ test defs: better detection of cross-compile mode
+ * configure.ac (AC_CANONICAL_HOST): New, probably not strictly
+ necessary, but useful to complements AC_CANONICAL_BUILD.
+ * tests/defs-static.in ($host_alias, $build_alias): New variables,
+ exported, user-overridable, defaults AC_SUBST'd by configure.
+ * tests/defs (cross_compiling): Consider cross-compilation mode
+ active only if `$host_alias' is not empty *and* it differs from
+ `$build_alias'. This is more faithful to the way configure
+ determines whether it is cross-compiling, but still allows the
+ user to easily force cross-compilation by overriding `host_alias'
+ and `build_alias' at runtime.
+ From a report by Ralf Wildenhues.
+
2011-05-20 Stefano Lattarini <stefano.lattarini@gmail.com>
testsuite: avoid re-running few tests with 'parallel-tests' option
and 'ANSI2KNR' in the example; instead, use the more common
and typical `DESTDIR'.
+2011-05-18 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: avoid spurious failures in cross-compile mode
+ * tests/depcomp2.test: Ensure verbose printing of captured stderr
+ from configure.
+ * tests/ansi3.test ($required): Add 'native', as the ansi2knr
+ functionality is not meant to work with a cross-compiler.
+ * tests/ansi3b.test: Likewise.
+ * tests/ansi5.test: Likewise.
+ * tests/ansi6.test: Likewise.
+ * tests/ansi7.test: Likewise.
+ * tests/ansi8.test: Likewise.
+ * tests/ansi10.test: Likewise.
+ * tests/subobj3.test: Likewise.
+ * tests/check8.test ($required): Add 'native', as this test is
+ not meant to work with a cross-compiler.
+ * tests/gnits2.test: Likewise.
+ * tests/ltconv.test: Likewise.
+ * tests/ltcond2.test: Likewise.
+ * tests/parallel-tests5.test: Likewise.
+ * tests/specflg7.test: Likewise.
+ * tests/specflg8.test: Likewise.
+ * tests/check5.test: Likewise. Also, ensure verbose printing of
+ captured make output, and thrown in few cosmetic and consistency
+ improvements.
+ * tests/ltinit.test: Be laxer in grepping configure output, to
+ avoid spurious failures on systems which lack POSIX dynamic
+ linking (e.g., MinGW), or when cross-compiling for such systems.
+ * tests/ansi9.test: Skip parts of the test that would require a
+ native compiler.
+ * tests/backcompat6.test: Likewise.
+ * tests/depcomp8a.test: Likewise.
+ * tests/pr401.test: Likewise.
+ * tests/pr401b.test: Likewise.
+ * tests/pr401c.test: Likewise.
+ * tests/target-cflags.test: Likewise.
+ * tests/lex3.test: Likewise.
+ * tests/yacc-basic.test: Likewise.
+ * tests/autohdr4.test: Likewise. Also, extend the test to ensure
+ more coverage in cross-compile mode.
+ * tests/canon7.test: Likewise.
+ * tests/check9.test: Removed, split into two new tests ...
+ * tests/check-subst-prog.test: ... this one, which requires a
+ native C compiler ...
+ * tests/check-subst.test: ... and this one, which doesn't require
+ any compiler.
+ * tests/parallel-tests4.test: Removed, split into two new tests ...
+ * tests/parallel-tests-suffix-prog.test: ... this one, which
+ requires a native C compiler ...
+ * tests/parallel-tests-suffix.test: ... and this one, which
+ doesn't require any compiler.
+ * tests/parallel-tests7.test: Removed, split into two new tests ...
+ * tests/parallel-tests-ext-driver-prog.test: ... this one, which
+ requires a native C compiler ...
+ * tests/parallel-tests-ext-driver.test: ... and this one, which
+ doesn't require any compiler.
+ * tests/Makefile.am (TESTS): Update.
+
2011-05-19 Stefano Lattarini <stefano.lattarini@gmail.com>
test defs: rename requirement 'non-cross' -> 'native'
2011-05-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+ tests: fix spurious failure of extradep.test on FreeBSD
+ * tests/extradep.test: When using `ls -t' to determine whether a
+ file has been updated, make sure to use as reference a file whose
+ timestamp is expected to be *strictly* older that that of the file
+ being checked. This is required because at least FreeBSD `ls' do
+ not sort files with the same timestamp in alphabetical order when
+ using the `-t' option.
+ * tests/extradep2.test: Likewise.
+
+2011-05-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
tests: fix spurious failure of txinfo21.test on FreeBSD
* tests/txinfo21.test: Use the `is_newest' subroutine instead of
the `ls -t' hack to to determine whether a file has been updated.
2011-05-07 Stefano Lattarini <stefano.lattarini@gmail.com>
- tests: fix spurious failure of extradep.test on FreeBSD
- * tests/extradep.test: When using `ls -t' to determine whether a
- file has been updated, make sure to use as reference a file whose
- timestamp is expected to be *strictly* older that that of the file
- being checked. This is required because at least FreeBSD `ls' do
- not sort files with the same timestamp in alphabetical order when
- using the `-t' option.
- * tests/extradep2.test: Likewise.
+ tests: fix syntax error in 'silentcxx-gcc.test'
+ * tests/silentcxx-gcc.test (for config_args in ...): Remove extra
+ trailing backslash from the list of looped-on items.
+
+2011-05-07 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ depmod tests: fix bug in depmodes list extraction
+ * tests/depmod-tests.sh (get_depmodes): Use `echo $all_depmodes'
+ instead of `... | tr "$nl" "$sp"' to strip extra whitespaces and
+ newlines from `$all_depmodes'. The latter idiom with `tr' is
+ wrong since it gets run before `./defs' is sourced, and thus `$nl'
+ and `$sp' are undefined.
+ Bug revealad by FreeBSD tr(1) implementation, which doesn't accept
+ empty strings as arguments.
+
+2011-05-04 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: add excerpts from few original bug reports, for clarity.
+ * tests/comment6.test: Add excerpts and/or details from the
+ original bug report that prompted this tests to be added and/or
+ extended.
+ * tests/dejagnu4.test: Likewise.
+ * tests/installdir.test: Likewise.
2011-05-04 Stefano Lattarini <stefano.lattarini@gmail.com>
implementation, looking for a generic `lex' program, will follow
in the future.
+2011-05-04 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: 'silent-many-gcc.test' improved and made more robust
+ * tests/silent-many-gcc.test: Also force "fast" gcc depmode
+ for C++ compilations. Add sanity checks verifying that the
+ cache variables we force are really used by configure. Fix
+ typo in comments.
+
+2011-05-04 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: drop useless requirements of 'g++'
+ * tests/lflags2.test ($required): Remove 'g++', as any working
+ C++ compiler should be enough, and ./configure will cause the
+ test to skip if no working C++ compiler is found.
+ * tests/yflags2.test: Likewise.
+ * tests/subobj9.test: Likewise.
+ * tests/silentcxx.test: Likewise. Also, do not force depmodes
+ that could cause non-GNU C++ compilers to fail.
+ * tests/silentcxx-gcc.test: New test, like `silentcxx.test',
+ but forcing "fast" gcc depmode (and thus requiring the GNU C++
+ compiler).
+ * tests/specflg10.test: Add proper "fixme" comment telling that
+ we should make this test work with a generic C++ compiler.
+ * tests/Makefile.am (TESTS): Update.
+
+2011-05-03 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests defs: one more environment sanitization (corner case)
+ * tests/defs (am__using_gmake): Initialize. The `using_gmake'
+ subroutine was using this variable for caching, but wasn't
+ initializing it, which could cause problems in the (admittedly
+ very unlikely) case in which it was pre-existent in the
+ environment.
+
2011-05-03 Stefano Lattarini <stefano.lattarini@gmail.com>
tests: add forgotten test scripts to $(TESTS)
2011-05-03 Stefano Lattarini <stefano.lattarini@gmail.com>
+ maintcheck: consistency of list of test scripts
+ * CheckListOfTests.am: New file.
+ (maintainer-check-list-of-tests): New target,
+ check for consistency between list of tests defined in the
+ including Makefile and list of tests on the filesystem.
+ (clean-maintcheck-testslist-tmp): New rule, to clean up the
+ temporary files that might be left around by the rules associated
+ with the previous target.
+ (clean-local): Depend on it.
+ * lib/Automake/tests/Makefile.am: Include `CheckListOfTests.am'.
+ (maintainer-check): Added dependency from the new target
+ `maintainer-check-list-of-tests'.
+ * tests/Makefile.am: Likewise. Also ...
+ (checked_test_extensions): Define to `.test'.
+ (expected_test_list): Define properly.
+ * Makefile.am (maintainer-check-list-of-test): New target,
+ calling recursively into `tests/' and `lib/Automake/tests/',
+ using ...
+ (TEST_SUBDIRS): ... this new variable.
+
+2011-05-03 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: drop useless requirements of gcc (plus testsuite tweakings)
+ * tests/libobj14.test ($required): Remove 'gcc', as we never run
+ the C compiler.
+ * tests/targetclash.test: Likewise.
+ * tests/ansi6.test ($required): Remove 'gcc', as any working
+ C compiler should be enough.
+ * tests/ansi7.test: Likewise.
+ * tests/ansi9.test: Likewise.
+ * tests/ansi10.test: Likewise.
+ * tests/lex5.test: Likewise.
+ * tests/lexvpath.test: Likewise.
+ * tests/mmodely.test: Likewise.
+ * tests/pr204.test: Likewise.
+ * tests/pr300-lib.test: Likewise.
+ * tests/pr300-prog.test: Likewise.
+ * tests/primary3.test: Likewise.
+ * tests/specflg7.test: Likewise.
+ * tests/specflg8.test: Likewise.
+ * tests/subdir5.test: Likewise.
+ * tests/subdir8.test: Likewise.
+ * tests/subobj6.test: Likewise.
+ * tests/subst3.test: Likewise.
+ * tests/substre2.test: Likewise.
+ * tests/yacc6.test: Likewise.
+ * tests/yacc8.test: Likewise.
+ * tests/depcomp2.test: Likewise. Also, avoid clobbering user-set
+ CFLAGS.
+ * tests/lex3.test: Likewise.
+ * tests/ansi3.test: Likewise. Also, avoid 'CC=gcc' in configure.
+ * tests/ansi3b.test: Likewise.
+ * tests/ansi5.test: Likewise.
+ * tests/autohdr4.test ($required): Remove 'gcc', as any working
+ C compiler should be enough.
+ Also, do not reject slow dependency extractors (which we might
+ be forced to use now that $CC is not necessarily gcc anymore).
+ * tests/cond16.test ($required): Remove 'gcc', as any working
+ C compiler should be enough.
+ Since we are at it, throw in few minor tweakings (mostly cosmetic,
+ stylistic, or consistency-related).
+ * tests/cond18.test: Likewise.
+ * tests/cond35.test: Likewise.
+ * tests/gnits2.test: Likewise.
+ * tests/libtool3.test: Likewise.
+ * tests/libtool7.test: Likewise.
+ * tests/libtool9.test: Likewise.
+ * tests/ltcond.test: Likewise.
+ * tests/ltcond2.test: Likewise.
+ * tests/ltconv.test: Likewise.
+ * tests/ltlibsrc.test: Likewise.
+ * tests/nobase.test: Likewise.
+ * tests/nobase-libtool.test: Likewise.
+ * tests/pr220.test: Likewise.
+ * tests/pr224.test: Likewise.
+ * tests/pr300-ltlib.test: Likewise.
+ * tests/pr401.test: Likewise.
+ * tests/pr401b.test: Likewise.
+ * tests/pr401c.test: Likewise.
+ * tests/subpkg.test: Likewise.
+ * tests/target-cflags.test: Likewise.
+ * tests/transform.test: Likewise.
+ * tests/yacc4.test: Likewise.
+ * tests/cond19.test: Likewise. Also, avoid clobbering user-set
+ CFLAGS.
+ * tests/cond4.test: Likewise.
+ * tests/depend2.test: Likewise.
+ * tests/pr87.test: Likewise.
+ * tests/subobj3.test: Likewise.
+ * tests/substref.test: Likewise.
+
+2011-05-03 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ test defs: increase coverage w.r.t. GNU make
+ * tests/defs (GNUmake): Instead of just skipping the tests
+ requiring GNU make if $MAKE is not GNU make, try to look for
+ it and, if found, redefine $MAKE accordingly. This will help
+ to transparently increase coverage on non-GNU systems which
+ have GNU make available in PATH.
+
+2011-05-03 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: improve few inter-tests references
+ * tests/acloca22.test: Improve and extend the heading comments.
+ Add reference to related tests 'remake-deleted-m4-file.test' and
+ 'remake-renamed-m4-macro-and-file.test'. Since we are at it,
+ add trailing `:' command, and few blank lines for readability.
+ * tests/remake-renamed-m4-macro-and-file.test: In the heading
+ comments, add reference to the related test 'acloca22.test'.
+
+2011-05-03 Stefano Lattarini <stefano.lattarini@gmail.com>
+
tests: fix spurious failure in 'color2.test'
* tests/color2.test: If $MAKE contains command-line arguments (as
in e.g., "make -j2"), expect's directive "spawn $env(MAKE)" fails
spuriously, because it tries to run "$MAKE" as a single command.
Fix this with proper uses of the TCL `eval' builtin.
+2011-05-03 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: few more checks on automatic remake support
+ * tests/remake-all-1.test: New test, check that the "all" target
+ triggers rebuilt of outdated Makefiles.
+ * tests/remake-all-2.test: Likewise, but for when the makefiles
+ are not named `Makefile'.
+ * tests/Makefile.am (TESTS): Update.
+
+2011-05-03 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: check some remake features with non-GNU make too
+ * tests/acloca14.test ($required): Drop "GNUmake". This test
+ works as-is with non-GNU make implementations.
+ * tests/remake1a.test: Likewise.
+ * tests/aclocal4.test ($required): Drop "GNUmake".
+ Modify the test to have it work also with non-GNU make.
+ * tests/remake5.test: Likewise.
+ * tests/remake8a.test: Likewise.
+ * tests/remake8b.test: Likewise.
+ * tests/remake9a.test: Likewise.
+ * tests/remake9b.test: Likewise.
+ * tests/remake9c.test: Likewise.
+ * tests/remake9d.test: Likewise.
+ * tests/remake10a.test: Likewise.
+ * tests/remake10b.test: Likewise.
+ * tests/remake10c.test: Likewise.
+ * tests/remake12.test: ($required): Drop "GNUmake".
+ Adapt the test to make it work also with non-GNU make (if
+ it supports an "include" directive).
+
+2011-05-01 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: extend tests on dependency tracking with VPATH
+ * tests/depmod-tests.sh: New script, fulfilling a threefold role:
+ 1. it is called to generate a Makefile.am snippet, containing the
+ definition of proper lists of tests;
+ 2. it is called to set up a directory containing some common data
+ files and autotools-generated files used by the aforementioned
+ tests (this is done for speed reasons only); and
+ 3. it is called to properly run those tests, one at a time.
+ * tests/depcomp9.test: Delete, it's obsolete now.
+ * tests/depcomp10.test: Likewise.
+ * tests/Makefile.am ($(srcdir)/depmod-tests.am): Include this
+ snippet, which defines ...
+ (depmode_tests): ... this new macro, containing the list of the
+ newly generated `*.depmod' tests.
+ (TESTS_EXTENSIONS): Add `.depmod'.
+ (DEPMOD_LOG_COMPILER): Define. It calls `depmod-tests.sh', so that
+ the "depmode tests" will be executed by passing that driver script
+ a proper parameter.
+ ($(depmod_tests)): New dependency declaration (dummy, but required
+ in order to have make actually produce expected log files from the
+ `.depmod.log' suffix rule).
+ (TESTS): Add $(depmod_tests).
+ (EXTRA_DIST): Distribute depmod-tests.sh.
+ Other minor cosmetic changes and reorderings.
+ * bootstrap: Generate depmod-tests.am.
+ * tests/.gitignore: Updated.
+
+2011-05-01 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: avoid extra test generation (for `instspc' tests)
+ We don't really need to generate tons and tons of wrapper test
+ scripts (presently, ~ 90) just to run what is basically a single
+ test (specifically, `instspc-tests.sh') with different parameters.
+ In fact, the possibility of running a single driver script with
+ multiple files/configurations/parameters is one of the major
+ selling points of the 'parallel-tests' testsuite driver of
+ automake. So why not use ourselves more extensively the best
+ features we provide?
+ * tests/instspc-tests.sh: Update heading comments. Expect
+ to be executed, not sourced. Update handling of command-line
+ arguments. Adjust to reflect the renaming of "instspc tests"
+ from `instspc-*.test' to `*.instspc'.
+ * tests/instspc-data.test: Update heading comments. Execute
+ driver script `instspc-tests.sh', instead of sourcing it.
+ Give more informative messages in case of internal errors.
+ * tests/Makefile.am (TESTS_EXTENSIONS): Define explicitly (to
+ `.test' and `.instspc').
+ (TESTS): Add `$(instspc_tests)'.
+ (generated_tests): Remove `$(instspc_tests)'.
+ (INSTSPC_LOG_COMPILER): Define. Calls `instspc-tests.sh', so
+ that the "instspc tests" will be now executed by passing that
+ driver script a proper parameter.
+ ($(instspc_tests)): Remove, we don't need anymore to generate
+ this tests.
+ ($(instspc_tests:.test=.log)): Remove, substituted by ...
+ ($(instspc_tests:.instspc=.log)): ... this.
+ ($(instspc_tests)): New dependency declaration (dummy, but
+ required in order to have make actually produce expected log
+ files from the `.instspc.log' suffix rule).
+ (MAINTAINERCLEANFILES, generated_tests): Don't extend with
+ $(instspc_tests) anymore.
+ Update comments.
+
+2011-05-01 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ build: improve the definition of the list of testcases
+ * tests/Makefile.am (handwritten_tests): New variable.
+ (generated_tests): Likewise.
+ (TESTS): Redefine as the union of the above.
+ (EXTRA_DIST): Extend using $(handwritten_tests) and
+ $(generated_tests) rather than $(TESTS).
+ * tests/gen-parallel-tests: Update accordingly, and
+ make more robust.
+
+2011-05-01 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests skips: shorter and clearer messages
+ * tests/defs: Use shorter messages when giving reasons for test
+ skipping; it turns out these shorter messages are also clearer.
+ If more info might be useful, send them to the log file only.
+
2011-05-01 Stefano Lattarini <stefano.lattarini@gmail.com>
tests: fix self check spurious failure with older bash versions
and 3.2.39) fail to correctly remove the temporary directory in
the exit trap.
+2011-05-01 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ tests: add sanity check to 'self-check-cleanup.test'
+ * tests/self-check-cleanup.test: Check that the "hacked" file
+ `defs-static' used by the test differs from the "vanilla" one
+ in builddir. This also offers a little more debugging output.
+
+2011-05-01 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: display reasons for skips to the console
+ * tests/Makefile.am (AM_TESTS_ENVIRONMENT): Open file descriptor
+ `9' to the original stderr; define `stderr_fileno_' to `9', and
+ export it.
+ * tests/self-check-report.test: Prevent new spurious failures by
+ removing from the environment any definition of `stderr_fileno_'.
+
+2011-05-01 Stefano Lattarini <stefano.lattarini@gmail.com>
+
+ testsuite: use `skip_' for skipping of tests
+ * tests/defs: Use the `skip_' subroutine for test skipping. Also
+ give sometimes more detailed messages about the reasons for the
+ skipping.
+ * tests/canon7.test: Likewise.
+ * tests/color.test: Likewise.
+ * tests/color2.test: Likewise.
+ * tests/compile2.test: Likewise.
+ * tests/dejagnu7.test: Likewise.
+ * tests/depcomp6.test: Likewise.
+ * tests/depcomp9.test: Likewise.
+ * tests/depcomp10.test: Likewise.
+ * tests/distlinks.test: Likewise.
+ * tests/distlinksbrk.test: Likewise.
+ * tests/fn99.test: Likewise.
+ * tests/fn99subdir.test: Likewise.
+ * tests/forcemiss2.test: Likewise.
+ * tests/fort5.test: Likewise.
+ * tests/gettext3.test: Likewise.
+ * tests/install2.test: Likewise.
+ * tests/instfail-info.test: Likewise.
+ * tests/instfail-java.test: Likewise.
+ * tests/instfail-libtool.test: Likewise.
+ * tests/instfail.test: Likewise.
+ * tests/instmany-mans.test: Likewise.
+ * tests/instmany-python.test: Likewise.
+ * tests/instmany.test: Likewise.
+ * tests/instsh3.test: Likewise.
+ * tests/ltinit.test: Likewise.
+ * tests/makej2.test: Likewise.
+ * tests/mdate6.test: Likewise.
+ * tests/mkinst3.test: Likewise.
+ * tests/parallel-tests3.test: Likewise.
+ * tests/parallel-tests-reset-term.test: Likewise.
+ * tests/parallel-tests-unreadable-log.test: Likewise,
+ * tests/python-virtualenv.test: Likewise.
+ * tests/remake-gnulib-remove-header.test: Likewise.
+ * tests/subobj9.test: Likewise.
+ * tests/symlink2.test: Likewise.
+ * tests/tar.test: Likewise.
+ * tests/tar2.test: Likewise.
+ * tests/txinfo26.test: Likewise.
+ * tests/vala2.test: Likewise.
+ * tests/vala3.test: Likewise.
+ * tests/vala5.test: Likewise.
+ * tests/vtexi4.test: Likewise.
+ * tests/instdir-texi.test: Likewise.
+ * tests/txinfo21.test: Likewise.
+
2011-04-29 Stefano Lattarini <stefano.lattarini@gmail.com>
testsuite: more environment sanitization
projects / platform / upstream / automake.git / commitdiff