platform/upstream/automake.git
13 years agotest defs: fix ksh-related portability bug in warning messages
Stefano Lattarini [Mon, 13 Jun 2011 20:42:25 +0000 (22:42 +0200)]
test defs: fix ksh-related portability bug in warning messages

Running "make check" normally prints a diagnostic to the outermost
stderr (usually a tty) to explain why a test is skipped, thus
giving better and faster feedback to the user.  It used to do
so by redirecting file descriptor 9 to stderr (via "exec 9>&2")
before invoking the test scripts, which then would write any skip
explanation to file descriptor 9 via the `skip_' function defined
in `tests/defs'.

However, various Korn Shells (at least Solaris 10's /bin/ksh and
Debian GNU/Linux's /bin/ksh) and the HP-UX's /bin/sh close open
file descriptors > 2 upon an `exec' system call; thus the effects
of "exec 9>&2" are cancelled upon fork-and-exec, so we would get
a "Bad file number" diagnostic and no skip explanation with those
shells.

The present change remedies this situation.

* tests/Makefile.am (AM_TESTS_ENVIRONMENT): Redirect more portably,
via a trailing "9>&2", rather than the prior "exec 9>&2; ...".  Add
explanatory comments.
* tests/defs (stderr_fileno_): Update the advice in comments.

Based on commit v8.12-82-g6b68745 "tests: accommodate HP-UX and
ksh-derived shells" in GNU coreutils.

Further references, with lots of discussion:
 <http://lists.gnu.org/archive/html/bug-autoconf/2011-06/msg00002.html>
 <http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/22488>
 <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=8846>

13 years agotests: don't hard-code the test name in txinfo21.test
Stefano Lattarini [Mon, 13 Jun 2011 16:02:40 +0000 (18:02 +0200)]
tests: don't hard-code the test name in txinfo21.test

* tests/txinfo21.test: Use `$me' instead of hard-coding the
current testcase name "txinfo21".  Add a trailing `:' command
since we are at it.

13 years agoMerge branch 'extend-coverage-for-add-missing' into testsuite-work
Stefano Lattarini [Sun, 12 Jun 2011 22:00:53 +0000 (00:00 +0200)]
Merge branch 'extend-coverage-for-add-missing' into testsuite-work

* extend-coverage-for-add-missing:
  tests: new test dedicated to `--add-missing' and `--copy'

13 years agoMerge branch 'master' into testsuite-work
Stefano Lattarini [Sat, 11 Jun 2011 10:54:44 +0000 (12:54 +0200)]
Merge branch 'master' into testsuite-work

13 years agoMerge branch 'maint'
Stefano Lattarini [Sat, 11 Jun 2011 10:47:56 +0000 (12:47 +0200)]
Merge branch 'maint'

* maint:
  test defs: new function 'fatal_', for hard errors

13 years agotestsuite: use 'fatal_' and 'framework_failure_' for hard errors
Stefano Lattarini [Tue, 7 Jun 2011 14:36:40 +0000 (16:36 +0200)]
testsuite: use 'fatal_' and 'framework_failure_' for hard errors

* tests/defs (require_xsi): Use `fatal_', not `framework_failure',
to report an invalid usage.
* tests/remake-gnulib-remove-header.test: Prefer using `fatal_'
with a proper error message over a direct call to `Exit 99'.
* tests/pr8365-remake-timing.test: Likewise.
* tests/cygnus-imply-foreign.test: Likewise.
* tests/missing6.test: Likewise.
* tests/cond8.test: Likewise.
* tests/cond33.test: Likewise.
* tests/python-virtualenv.test: Prefer using `framework_failure_'
with a proper error message over a direct call to `Exit 99'.
* tests/instspc-tests.sh: Prefer using `framework_failure_' and
`fatal_' over direct calls to `Exit 99'.
(fatal_): Define this (which is a simplified version of the one
in `tests/defs') for early uses (i.e., before `tests/defs'
gets sourced).
* tests/depmode-tests.sh: Likewise.  Also, simplify the
'get_depmodes' function and calls to it accordingly.

13 years agoself tests: check new 'fatal_' function
Stefano Lattarini [Tue, 7 Jun 2011 14:00:31 +0000 (16:00 +0200)]
self tests: check new 'fatal_' function

* tests/self-check-exit.test: Also check the new 'fatal_'
function.

13 years agoMerge branch 'maint' into testsuite-work
Stefano Lattarini [Sat, 11 Jun 2011 10:22:15 +0000 (12:22 +0200)]
Merge branch 'maint' into testsuite-work

* maint:
  test defs: new function 'fatal_', for hard errors

13 years agotests: new test dedicated to `--add-missing' and `--copy'
Stefano Lattarini [Tue, 7 Jun 2011 20:49:28 +0000 (22:49 +0200)]
tests: new test dedicated to `--add-missing' and `--copy'

* tests/add-missing.test: New test.
* tests/Makefile.am (TESTS): Update.

Suggested by Peter Rosin.

13 years agotest defs: new function 'fatal_', for hard errors
Stefano Lattarini [Tue, 7 Jun 2011 13:24:11 +0000 (15:24 +0200)]
test defs: new function 'fatal_', for hard errors

Before this patch, the only way offered by tests/defs to
properly signal a hard error was the `framework_failure_'
function.  But the error message issued by that function,
as its name would suggest, refers to a set-up failure in the
testsuite, while hard errors can obviously also be due to
other reasons.  The best way to fix this inconsistency is to
introduce a new function with a more general error message.

Inspired by a recent similar change to Gnulib's tests/init.sh.

* tests/defs.in (fatal_): New function.
* tests/README (Section "Writing test cases" subsection "Do"):
Suggest the use of `fatal_', not of `framework_failure_', for
generic hard errors.  The latter should be reserved for "real"
set-up failures.

13 years agoMerge branch 'tests-more-shells' into testsuite-work
Stefano Lattarini [Wed, 8 Jun 2011 08:19:18 +0000 (10:19 +0200)]
Merge branch 'tests-more-shells' into testsuite-work

13 years agoMerge branch 'master' into testsuite-work
Stefano Lattarini [Wed, 8 Jun 2011 08:03:52 +0000 (10:03 +0200)]
Merge branch 'master' into testsuite-work

13 years agoMerge branch 'fix-auxdir2-test'
Stefano Lattarini [Wed, 8 Jun 2011 08:03:29 +0000 (10:03 +0200)]
Merge branch 'fix-auxdir2-test'

13 years agoself tests: check that `$me' can be overridden
Stefano Lattarini [Sun, 5 Jun 2011 18:58:21 +0000 (20:58 +0200)]
self tests: check that `$me' can be overridden

* tests/self-check-me.test: Check that `$me' can be overridden
before sourcing ./defs, with or without sourcing ./defs-static
beforehand, and that this override is honored.  Update heading
comments.

13 years agotests: `lib/' shell scripts transparently tested also with $SHELL
Stefano Lattarini [Mon, 6 Jun 2011 14:12:59 +0000 (16:12 +0200)]
tests: `lib/' shell scripts transparently tested also with $SHELL

With the previous commit, the user could prefer the use of $SHELL
over /bin/sh in some tests checking the Automake-provided shell
scripts by manually exporting `test_prefer_config_shell' to "yes"
in the environment.  With this commit, we ensure that such tests
*always* and *transparently* run using both $SHELL and /bin/sh
to execute the checked scripts.  The `test_prefer_config_shell'
variable becomes an internal detail, and is no more meant to be
manually defined or overridden.

* tests/defs-static.in: Check that `test_prefer_config_shell' is
not exported in the environment.  Error out if this is the case.
* tests/config-shell-tests.sh: New file, driver script to run
checks on the shell scripts in `lib/' using the $SHELL determined
at configure time instead of the default system shell /bin/sh.
* Makefile.am (TESTS_EXTENSIONS): Add `.shtst'.
(SHTST_LOG_COMPILER): Define, it calls `config-shell-tests.sh'.
(config_shell_tests): Define to a list of tests that wraps other
`*.test' tests using `config-shell-tests.sh'.
($(config_shell_tests)): Dummy dependency declaration required
in order to have make actually produce expected log files from
the `.shtst.log' suffix rule.
(EXTRA_DIST): Distribute `config-shell-tests.sh'.
(TESTS): Add `$(config_shell_tests)'.
* tests/self-check-env-sanitize.test: Update, by checking that
`test_prefer_config_shell' isn't exported in the environment.

13 years agotests: can use also $SHELL to check shell scripts from `lib/'
Stefano Lattarini [Mon, 6 Jun 2011 12:40:22 +0000 (14:40 +0200)]
tests: can use also $SHELL to check shell scripts from `lib/'

* tests/ar-lib.test: If the variable `$test_prefer_config_shell'
is set to "yes", run the script under test with configure-time
determined $SHELL, rather than with /bin/sh.
The `$test_prefer_config_shell' variable defaults to empty, but
can be overridden at runtime by the user, thus allowing more
coverage.
* tests/compile.test: Likewise.
* tests/compile2.test: Likewise.
* tests/compile3.test: Likewise.
* tests/compile4.test: Likewise.
* tests/compile5.test: Likewise.
* tests/compile6.test: Likewise.
* tests/instsh2.test: Likewise.
* tests/instsh3.test: Likewise.
* tests/mkinst3.test: Likewise.
* tests/missing.test: Likewise.
* tests/missing2.test: Likewise.
* tests/missing3.test: Likewise.
* tests/missing5.test: Likewise.
* tests/defs (get_shell_script): New subroutine, factoring out
code common to the tests above.
(xsi-lib-shell): If `$test_prefer_config_shell' is set to "yes",
check that $SHELL, not /bin/sh, supports XSI constructs, as we
expect the test will use $SHELL and not /bin/sh to run the
script being tested.

13 years agotests defs: better requirements for XSI shells
Stefano Lattarini [Mon, 6 Jun 2011 09:32:19 +0000 (11:32 +0200)]
tests defs: better requirements for XSI shells

This change avoids potential spurious failures with tests using
the requirement 'xsi-shell' to mean that they want */bin/sh* (not
$SHELL) to be XSI-conforming.  This idiom used to work before
commit `v1.11-874-g1321be7' (as back then the test scripts were
unconditionally run with /bin/sh), but has become inconsistent
now that the test scripts re-execute themselves with configure
determined $SHELL.

The described spurious failures have already occurred in practice,
for examples on Solaris systems which also had GNU Bash installed.

From a suggestion by Peter Rosin.  See discussion at:
<http://lists.gnu.org/archive/html/automake-patches/2011-06/msg00016.html>

* tests/defs (xsi-shell): Now check that $SHELL, rather than the
shell currently running the test script, is an XSI shell.
(xsi-bin-sh): New requirement, checking that /bin/sh (which can
differ from $SHELL) is an XSI shell.
(xsi-lib-shell): New requirement, checking that the shell that
should be used to test the Automake-provided scripts from `lib/'
is an XSI shell.  For the moment, this is just an alias for
`xsi-bin-sh'.
(require_xsi): New subroutine, used to factor out code common to
the requirements above.
($xsi_shell_code): New variable, contains shell code supposed to
work only with XSI shells.  Used by the new subroutine above.
* tests/ar-lib.test ($required): Require 'xsi-lib-shell' instead
of 'xsi-shell', since the script we test here is run with /bin/sh,
not with $SHELL.
* tests/compile3.test: Likewise.
* tests/compile6.test: Likewise.

13 years agotests: fix typo-related error in auxdir2.test
Stefano Lattarini [Sun, 5 Jun 2011 19:44:53 +0000 (21:44 +0200)]
tests: fix typo-related error in auxdir2.test

* tests/auxdir2.test (configure.in):  Close m4 quoting in the
argument to AC_CONFIG_AUX_DIR.  Without this, aclocal fails with
"ERROR: end of file in string".  This problem hasn't been exposed
by the testsuite before because this test is in XFAIL_TESTS, so
its failure went unnoticed, even if it was due to a wrong cause.

Bug introduced in commit v1.11-249-g49ac3de.

13 years agoRevert "tests: use `$SHELL' to run the shell scripts from `lib/'"
Stefano Lattarini [Sun, 5 Jun 2011 10:16:48 +0000 (12:16 +0200)]
Revert "tests: use `$SHELL' to run the shell scripts from `lib/'"

This reverts commit f977d00e0a3ab9af3555bb3bc8ea78726cb7a143.

See <http://lists.gnu.org/archive/html/automake-patches/2011-06/msg00018.html>
for the reasons behind this revert.

13 years agoself tests: fix another spurious failure
Stefano Lattarini [Thu, 2 Jun 2011 15:54:25 +0000 (17:54 +0200)]
self tests: fix another spurious failure

Our ad-hoc usage of `tests/defs' in the testsuite's self tests
stopped working properly when we made the test scripts re-execute
themselves with the configure-time $SHELL.  Fix this.

Fixes a regression introduced by commit 'v1.11-874-g1321be7'.

* tests/self-check-exit.test: Export `AM_TESTS_REEXEC' to "no"
before running the self tests.  This fixes a spurious failure
present only when the test was run by hand.

13 years agotests: use `$SHELL' to run the shell scripts from `lib/'
Stefano Lattarini [Thu, 2 Jun 2011 15:05:51 +0000 (17:05 +0200)]
tests: use `$SHELL' to run the shell scripts from `lib/'

This should offer greater testsuite coverage for those developers
that override CONFIG_SHELL at configure time in order to test more
shells on a single system, instead of just the default `/bin/sh'.

This change also fixes few spurious failures in tests using the
`xsi-shell' requirement, where inconsistencies could crop up if
the shell probed for XSI features (which, by default, is $SHELL)
was not the same shell later used to run the scripts using those
features (which was hard-coded to `/bin/sh').  Such failures have
already occurred in practice, for examples on Solaris systems
which had also GNU Bash installed.

* tests/ar-lib.test: Run the `ar-lib' script with `$SHELL', rather
than directly with `./ar-lib', which would make run unconditionally
with `/bin/sh'.
* tests/compile.test: Likewise, but for the `compile' script.
* tests/compile2.test: Likewise.
* tests/compile3.test: Likewise.
* tests/compile4.test: Likewise.
* tests/compile5.test: Likewise.
* tests/compile6.test: Likewise.
* tests/instsh2.test: Likewise, but for the `install' script.
* tests/instsh3.test: Likewise.
* tests/mkinst3.test: Likewise, but for the `mkinstalldirs' script.
* tests/missing.test: Likewise, but for the `missing' script.
* tests/missing2.test: Likewise.
* tests/missing3.test: Likewise.
* tests/missing5.test: Likewise.

13 years agosilent-rules tests: fix spurious failures with Sun Studio C++
Stefano Lattarini [Thu, 2 Jun 2011 13:35:03 +0000 (15:35 +0200)]
silent-rules tests: fix spurious failures with Sun Studio C++

* tests/silentcxx.test: The C++ compiler from Sun Studio is named
`CC'.  Account for this in our grepping checks on the make output.
Since we are at it, throw in a couple of improvements to comments
and formatting.
* tests/silent-many-generic.test: The C++ compiler from Sun Studio
is named `CC', and this can cause spurious failures in our grepping
of the make output.  Work around this by using a wrapper script
around the C++ compiler (generated on the fly), since filtering the
make output proved to be too fragile.

13 years agotests: fix spurious failure in backcompat2.test on NetBSD
Stefano Lattarini [Thu, 2 Jun 2011 12:40:19 +0000 (14:40 +0200)]
tests: fix spurious failure in backcompat2.test on NetBSD

* tests/backcompat2.test: Add trailing `:' in the body of a `for'
loop, in case the last command there might have an exit status
different 0.  This work around `set -e' issues in some BSD shells,
e.g., NetBSD /bin/ksh.

13 years agotests: fix spurious failure in autohdr3.test
Stefano Lattarini [Thu, 2 Jun 2011 12:14:28 +0000 (14:14 +0200)]
tests: fix spurious failure in autohdr3.test

* tests/autohder3.test (Makefile.am): Let `test' depend on `all',
so that config.h header is truly remade.
Remove now useless call to `$MAKE' when non-GNU make is in use.

Failure introduced by commit `v1.11-895-g5e62b96'.

13 years agotests: fix spurious failure of cond29.test on NetBSD
Stefano Lattarini [Thu, 2 Jun 2011 12:13:18 +0000 (14:13 +0200)]
tests: fix spurious failure of cond29.test on NetBSD

* tests/cond29.test: Skip if we cannot safely limit the maximal
size of used virtual memory to 20K.  This fixes a spurious failure
on NetBSD.

Bug introduced in commit `v1.11-885-g908d335'.

13 years agomaintcheck: fix again few more failures
Stefano Lattarini [Thu, 2 Jun 2011 10:58:37 +0000 (12:58 +0200)]
maintcheck: fix again few more failures

* tests/cond29.test: To please the `sc_tests_Exit_not_exit'
maintainer check, avoid using `sh -c "exit 0"' where a simple
`sh -c :' will do.
* tests/depmod-data.test: Use creative quoting to avoid
spuriously triggering the `sc_tests_Exit_not_exit' maintainer
check.

13 years agoMerge branch 'master' into testsuite-work
Stefano Lattarini [Thu, 2 Jun 2011 10:54:03 +0000 (12:54 +0200)]
Merge branch 'master' into testsuite-work

* master:
  maintcheck: fix some more failures
  maintcheck: fix some failures, extend some checks

13 years agomaintcheck: fix some more failures
Stefano Lattarini [Thu, 2 Jun 2011 10:47:58 +0000 (12:47 +0200)]
maintcheck: fix some more failures

* tests/instdir-ltlib.test: Use creative quoting to avoid
spuriously triggering the `sc_rm_minus_f' maintainer check.
* tests/instdir-prog.test: Likewise.
* tests/instspc-data.test: Use creative quoting to avoid
spuriously triggering the `sc_tests_Exit_not_exit' maintainer
check.

13 years agoMerge branch 'maint'
Stefano Lattarini [Thu, 2 Jun 2011 10:17:40 +0000 (12:17 +0200)]
Merge branch 'maint'

* maint:
  maintcheck: fix some failures, extend some checks

13 years agomaintcheck: fix some failures, extend some checks
Stefano Lattarini [Thu, 2 Jun 2011 10:15:52 +0000 (12:15 +0200)]
maintcheck: fix some failures, extend some checks

* Makefile.am (sc_diff_automake_in_automake): Update, as we
now expect 9 lines, not 8, to be changed from `automake.in'
to `automake'.
(sc_diff_aclocal_in_aclocal): New maintainer check, similar to
the above, and checking that only 10 lines are changed from
`aclocal.in' to `aclocal'.
(syntax_check_rules): Update.
(sc_tests_Exit_not_exit): Exempt self tests `self-check-*.test'
from this check, as they can legitimately use the bare `exit'
builtin in various places.
* doc/automake.texi (Python): Remove stray `@' from the end of
a line.  Typo introduced in commit `v1.11-312-g5bf7af6'.
* tests/depcomp8a.test: Pass DISTCHECK_CONFIGURE_FLAGS to make
from the environment rather than from the command line, to
pacify the `sc_tests_overriding_macros_on_cmdline' maintainer
check.
* tests/depcomp8b.test: Likewise.

13 years agotests: don't require GNU make where it's not strictly needed
Stefano Lattarini [Thu, 2 Jun 2011 09:22:32 +0000 (11:22 +0200)]
tests: don't require GNU make where it's not strictly needed

* tests/autohdr3.test: Rewrite to be stricter when make is GNU
make.  Drop the requirement of GNU make.
* tests/aclocal5.test: Drop GNU make requirement, it's not truly
needed.  Add reference to similar tests `remake-subdir*.test'.
* remake-subdir-gnu.test, remake-subdir-from-subdir.test,
remake-subdir.test, remake-subdir2.test: Add reference to
each other, and to related test `aclocal5.test'.
* tests/aclocal6.test: Drop GNU make requirement, it's not
truly needed.
* tests/confh6.test: Likewise.
* tests/lex3.test: Likewise.
* tests/remake11.test: Likewise.
* tests/subdir5.test: Likewise.
* tests/subdir8.test: Likewise.
* tests/werror2.test: Likewise.
* tests/conff.test: Likewise, and ensure verbose printing of
captured make output.
* tests/lex5.test: Tweak so that GNU make is no more required.
* tests/version7.test: Likewise.
* tests/maken2.test: Add explicative comment for why this test
required GNU make.
* tests/maken4.test: Let it run also with BSD makes supporting
the `.MAKE' special target.
* tests/output6.test: Use proper m4 quoting in configure.in.
Expand make macros with one-character name using `$(x)', not
`$x', for portability.  Move checks in the makefiles, rather
than relying on grepping the output from make.  Drop the now
unneeded GNU make requirement.

13 years agoMerge branch 'master' into testsuite-work
Stefano Lattarini [Wed, 1 Jun 2011 18:23:45 +0000 (20:23 +0200)]
Merge branch 'master' into testsuite-work

* master:
  remake: behave better with non-GNU make in subdirectories

13 years agoMerge branch 'maint'
Stefano Lattarini [Wed, 1 Jun 2011 17:37:18 +0000 (19:37 +0200)]
Merge branch 'maint'

* maint:
  remake: behave better with non-GNU make in subdirectories

13 years agoMerge branch 'remake-rules-non-GNU-make' into maint
Stefano Lattarini [Wed, 1 Jun 2011 16:58:35 +0000 (18:58 +0200)]
Merge branch 'remake-rules-non-GNU-make' into maint

13 years agolex tests: fix spurious failures with Solaris lex
Stefano Lattarini [Wed, 1 Jun 2011 16:49:27 +0000 (18:49 +0200)]
lex tests: fix spurious failures with Solaris lex

* tests/lex-lib.test (foo.l): Avoid empty "rules section", which
can confuse Solaris lex.
* tests/lex-libobj.test (foo.l): Likewise.

13 years agolex tests: do not force the use of flex unconditionally
Stefano Lattarini [Wed, 1 Jun 2011 15:55:05 +0000 (17:55 +0200)]
lex tests: do not force the use of flex unconditionally

* tests/defs (lex): Act more similarly to what the `yacc'
requirement does, i.e., only force the use of flex if the
$LEX variable is left unset by the user.
(flex): Use `skip_' to skip the test if flex is not found.

13 years agolex tests: avoid possible hang; fix and extend
Stefano Lattarini [Wed, 1 Jun 2011 15:43:44 +0000 (17:43 +0200)]
lex tests: avoid possible hang; fix and extend

* tests/lex3.test (foo.l:yywrap): Return 1, not 0, to avoid hangs.
Bug introduced in commit 'v1.11-871-geb147a1'.
(Makefile.am): Do not add `@LEXLIB@' to `$(LDADD)', as we define
our own `yywrap' function.
* tests/lex.test (tscan.l): In `yywrap', return 1, not 0, for
consistency with the default flex implementation.
* tests/lex-libobj.test (yywrap.c): Likewise.
* tests/lex-subobj-nodep.test (s1.l): Likewise.
* tests/lexvpath.test (foo.c): Likewise.
* tests/silent-lex-gcc (foo.l): Likewise.
* tests/silent-lex-generic (foo.l): Likewise.
* tests/silent-many-gcc (foo5.l): Likewise.
* tests/silent-many-generic (foo5.l): Likewise.
* tests/lex-lib.test (mu.c): Likewise.
Update heading comments, to refer to ...
* tests/lex-lib-external.test: ... this new test, which checks
that we can get use the `yywrap' function from a system-wide
library, if that's available.

13 years agotests: prefer `skip_' over `echo ...; Exit 77'
Stefano Lattarini [Wed, 1 Jun 2011 14:45:19 +0000 (16:45 +0200)]
tests: prefer `skip_' over `echo ...; Exit 77'

* tests/self-check-cleanup.test: When the test must be skipped,
use `skip_ REASON' instead of `echo REASON; Exit 77'.  Also,
make the skip message shorter and clearer.

13 years agotests: fix spurious failures in self tests
Stefano Lattarini [Wed, 1 Jun 2011 14:33:07 +0000 (16:33 +0200)]
tests: fix spurious failures in self tests

Our ad-hoc usage of `tests/defs' in the testsuite's self tests
stopped working properly when we made the test scripts re-execute
themselves with the configure-time $SHELL.  Fix this.

Fixes a bug introduced by commit 'v1.11-874-g1321be7'.

* tests/defs: Only check that we can find the client test script
when we must re-execute it.
* tests/self-check-cleanup.test: Export `AM_TESTS_REEXEC' to "no"
before running the self tests.
* tests/self-check-dir.test: Likewise.
* tests/self-check-explicit-skips.test: Likewise.
* tests/self-check-me.test: Likewise.
* tests/self-check-sanity.test: Likewise.
* tests/self-check-reexec.test: New test.
* tests/Makefile.am (TESTS): Update.

13 years agotests: prefer ulimit over timeout in test on a memory-hogging bug
Stefano Lattarini [Wed, 1 Jun 2011 10:09:23 +0000 (12:09 +0200)]
tests: prefer ulimit over timeout in test on a memory-hogging bug

* tests/cond29.test: Use 'ulimit' instead of 'timeout' to ensure
that automake does not uses up too much resources.  This is really
and improvement because the bug tested by this script wasn't just
a "it hangs" or "it runs too slow" bug, but rather a memory-hogging
bug (due to combinatorial explosion when many Automake conditionals
had to be handled) which could easily crash the whole system, which
is unacceptable.  The requirement of a proper and working 'ulimit'
builtin might cause the test to be skipped on more systems, but
that shouldn't be a problem since the bug isn't about a portability
issue, but is rather an automake internal implementation problem.
I've verified that the test as updated by this patch still passes
with automake 1.8.5, automake 1.10.2, and obviously the development
version of automake, and that it fails with automake 1.7.9.

13 years agotestsuite: avoid generating `*-p.test' tests, use a wrapper script
Stefano Lattarini [Tue, 31 May 2011 19:42:16 +0000 (21:42 +0200)]
testsuite: avoid generating `*-p.test' tests, use a wrapper script

The generated `*-p.test' tests had already become just thin layers
around the corresponding test scripts.  This change makes the final
step, converting to the use of a generic wrapper script and thus
avoiding the extra test generation (similarly to what is done for
the `*.instspc' and `*.depmod' tests).

* tests/parallel-tests.sh: New file, driver script to run checks
the on the `parallel-tests' semantics by wrapping tests that use
the generic "Simple Tests" driver.
* tests/gen-parallel-tests: Update, mostly to reflect the new
`.ptest' extensions used for tests in $(parallel_tests).
* Makefile.am (TESTS_EXTENSIONS): Add `.ptest'.
(PTEST_LOG_COMPILER): Define, it calls `parallel-tests.sh'.
($(parallel_tests)): Do not really generate `*-p.test' tests
anymore; this is now just a dummy dependency declaration required
in order to have make actually produce expected log files from
the `.ptest.log' suffix rule.
(EXTRA_DIST): Distribute `parallel-tests.sh'.
(MAINTAINERCLEANFILES): Don't remove the `$(parallel_tests)', it
is not necessary anymore.
(generated_tests): Variable definition removed.
(TESTS): Update, by listing `$(parallel_tests)' directly instead
of `$(generated_tests)'
(expected_list_of_tests): Remove `$(generated_tests)'.
(maintainer-check-list-of-tests): No need to explicitly depend on
`$(expected_list_of_tests)' anymore.

13 years agoMerge branch 'master' into testsuite-work
Stefano Lattarini [Tue, 31 May 2011 16:07:56 +0000 (18:07 +0200)]
Merge branch 'master' into testsuite-work

* master:
  automake, aclocal: honour configure-time AUTOCONF and AUTOM4TE
  build: the user can override AUTOM4TE, AUTORECONF and AUTOUPDATE too
  tests/README: fix example about `make -e' usage

13 years agoMerge branch 'maint'
Stefano Lattarini [Tue, 31 May 2011 16:03:48 +0000 (18:03 +0200)]
Merge branch 'maint'

* maint:
  automake, aclocal: honour configure-time AUTOCONF and AUTOM4TE
  build: the user can override AUTOM4TE, AUTORECONF and AUTOUPDATE too
  tests/README: fix example about `make -e' usage

13 years agoautomake, aclocal: honour configure-time AUTOCONF and AUTOM4TE
Stefano Lattarini [Sun, 29 May 2011 08:42:00 +0000 (10:42 +0200)]
automake, aclocal: honour configure-time AUTOCONF and AUTOM4TE

Currently, the Automake's own configure script allow definition
of AUTOCONF and AUTOM4TE, expected to point respectively to an
autoconf and autom4te programs.  But while these definitions are
honoured in the Automake's build systems and test suite, they
were *not* honoured in the generated `automake' and `aclocal'
scripts.  This behaviour, apart from being wrong in that it does
not allow the user enough freedom in choosing his tools, also
caused inconsistencies in the test suite, brining to spurious
failures.

Problem reported by Graham Reitz on the automake list; see thread:
<http://lists.gnu.org/archive/html/automake/2011-05/msg00022.html>

* automake.in ($traces): Use `@am_AUTOCONF', not simply `autoconf'.
* aclocal.in ($traces): Use `@am_AUTOM4TE', not simply `autom4te'.
* Makefile.am (do_subst): Substitute also `@am_AUTOCONF' and
`@am_AUTOM4TE'.
* NEWS: Update.
* THANKS: Update.

13 years agobuild: the user can override AUTOM4TE, AUTORECONF and AUTOUPDATE too
Stefano Lattarini [Sun, 29 May 2011 09:04:08 +0000 (11:04 +0200)]
build: the user can override AUTOM4TE, AUTORECONF and AUTOUPDATE too

Our build system allows the user to override AUTOCONF and AUTOHEADER
at configure time, and honours these overrides in our testsuite.
But it didn't do the same with AUTOM4TE, AUTORECONF and AUTOUPDATE.
This change fixes that inconsistency.

* configure.ac (am_AUTOM4TE, am_AUTOUPDATE, am_AUTORECONF): New
AC_SUBSTitutions.  Update comments.
* tests/defs.in ($AUTOUPDATE): Default to `@am_AUTOUPDATE@' now.
($AUTOM4TE): New variable, defaulting to `@am_AUTOM4TE@'.
($AUTORECONF): New variable, defaulting to `@am_AUTORECONF@'.
* doc/Makefile.am ($(srcdir)/amhello-1.0.tar.gz): Update.

13 years agoremake: behave better with non-GNU make in subdirectories
Stefano Lattarini [Sun, 29 May 2011 13:24:58 +0000 (15:24 +0200)]
remake: behave better with non-GNU make in subdirectories

Currently, with every decent make program, it is possible to
rebuild out-of-date autotools-generated files with a simple
"make Makefile" -- but for this to work reliably with non-GNU
make implementations, the command must be issued from the
top-level directory.  This patch removes such limitation.

* lib/am/configure.am (am--refresh): Depend on `%MAKEFILE%'.
* tests/defs.in (using_gmake): New function, backported from the
`master' branch (and simplified).
* tests/remake-subdir.test: New test.
* tests/remake-subdir2.test: Likewise.
* tests/remake-subdir-gnu.test: Likewise.
* tests/remake-subdir-from-subdir.test: Likewise.
* tests/Makefile.am (TESTS): Update.

13 years agotests: don't require gfortran if any fortran compiler is enough
Stefano Lattarini [Sun, 29 May 2011 16:13:11 +0000 (18:13 +0200)]
tests: don't require gfortran if any fortran compiler is enough

* tests/silentf90.test ($required): Require `fortran', not
`gfortran'.
* tests/silentf77.test ($required): Require `fortran77', not
`gfortran'.
* tests/silent-many-generic.test ($required): Require `fortran'
and `fortran77' rather than `gfortran'.

13 years agotests: improve `ccnoco*.test', better cross-compiling support
Stefano Lattarini [Fri, 27 May 2011 13:05:40 +0000 (15:05 +0200)]
tests: improve `ccnoco*.test', better cross-compiling support

* tests/ccnoco3.test (Mycomp): Use the `$CC' chosen by `tests/defs'
instead of forcing `gcc' unconditionally.  This ensures better
coverage in case of cross-compiling, when GCC can be named e.g.,
`i586-mingw32msvc-gcc', instead of simply `gcc'.
* tests/ccnoco.test: Likewise.  Remove redundant checks.  Modernize
the created `configure.in'.  Run tests both in-tree and in VPATH.
Export `CC' to the overridden value only once.
* tests/ccnoco2.test: Slightly stricter grepping of automake
stderr.  Add trailing `:' command.

13 years agoMerge branch 'master' into testsuite-work
Stefano Lattarini [Fri, 27 May 2011 08:22:31 +0000 (10:22 +0200)]
Merge branch 'master' into testsuite-work

13 years agotestsuite: each test case depends on `defs-static'
Stefano Lattarini [Fri, 27 May 2011 08:20:25 +0000 (10:20 +0200)]
testsuite: each test case depends on `defs-static'

* tests/Makefile.am ($(TEST_LOGS)): Depends on `defs-static' too.
Simplify comments.

13 years agotestsuite: use $SHELL to run tests which are shell scripts
Stefano Lattarini [Fri, 3 Dec 2010 14:24:14 +0000 (15:24 +0100)]
testsuite: use $SHELL to run tests which are shell scripts

* tests/Makefile.am (TEST_LOG_COMPILER): Define so that the
configure-time $SHELL is used to run the tests.
* tests/defs: Add code to re-execute by default the running
test script with configure-time $SHELL.  Updated comments.
* configure.ac: Check that `set -e' is working for $SHELL,
not for /bin/sh.
* tests/defs-static.in: Update comments.
* tests/README (Supported shells): Updated.
(Getting details from failures): Don't tell that tests are
run by /bin/sh by default.

13 years agotests: split 'subst2.test' to improve modularity and coverage
Stefano Lattarini [Thu, 26 May 2011 20:28:45 +0000 (22:28 +0200)]
tests: split 'subst2.test' to improve modularity and coverage

* tests/subst2.test: Test removed, split into ...
* tests/programs-primary-rewritten.test: ... this test ...
* tests/subst-no-trailing-empty-line.test: ... and this one ...
* tests/extra-programs-empty.test: ... and this one.
* tests/Makefile.am (TESTS): Update.

13 years agotests: tweak and improve tests on Automake conditionals
Stefano Lattarini [Thu, 26 May 2011 13:42:02 +0000 (15:42 +0200)]
tests: tweak and improve tests on Automake conditionals

The "avoid the requirement of a working compiler" we refer about
below is obtained by disabling automatic dependency tracking and
defining `CC' (and `EXEEXT', `OBJEXT', `CXX', etc., if needed) to
dummy values, either directly in the `Makefile.am', or by AC_SUBST
in `configure.in'.

The "cosmetic changes" we refer about below consists in, e.g.,
adding a trailing `:' command to a script, using proper m4 quoting
in `configure.in' files, adding commands that offer more debugging
output, explicitly declaring phony targets as such in Makefiles,
making use of idioms more consistent with those employed in other
tests, following the GNU formatting standards more closely, etc.

* tests/cond.test: Cosmetic changes.
* tests/cond3.test: Likewise.
* tests/cond10.test: Likewise.
* tests/cond14.test: Likewise.
* tests/cond15.test: Likewise.
* tests/cond17.test: Likewise.
* tests/cond40.test: Likewise.
* tests/cond13.test: Likewise, and make grepping of the generated
`Makefile.in' slightly stricter.
* tests/cond6.test: Extend by also running ./configure, make and
"make install".  Few cosmetic changes.
* tests/cond8.test: Likewise.
* tests/cond2.test: Do few cosmetic changes.  Make grepping of
automake stderr stricter.
* tests/cond20.test: Likewise.
* tests/cond23.test: Likewise.
* tests/cond24.test: Likewise.
* tests/cond42.test: Likewise.
* tests/cond46.test: Likewise.
* tests/cond9.test: Move more checks in the `Makefile.am' instead
of grepping make's output.
* tests/cond38.test: Likewise.
* tests/cond11.test: Likewise.  Avoid the requirement of a working
C compiler.
* tests/cond16.test: Likewise.
* tests/cond22.test: Likewise.
* tests/cond30.test: Likewise.
* tests/cond31.test: Likewise.
* tests/cond4.test: Likewise, and drop the now-useless GNU make
requirement as well.
* tests/cond18.test: Likewise.
* tests/cond19.test: Likewise.
* tests/cond22.test: Likewise.
* tests/cond25.test: Add trailing `:' command.
* tests/cond26.test: Likewise.
* tests/cond27.test: Likewise.
* tests/cond28.test: Likewise.
* tests/cond29.test: Use `unindent' function for better formatting.
Other minor cosmetic changes.  Use a `timeout' program (if it's
available) to determine whether the script takes too long, instead
of just hoping that the user will notice an abnormally long test
execution time.
* tests/cond33.test: Extend a bit.  Fix heading comments.  Few more
cosmetic changes.
* tests/cond37.test: Ensure verbose printing of captured make
output.  Minor cosmetic changes.
* tests/cond34.test: Likewise.  Avoid the requirement of a working
C compiler.  Make grepping of make output slightly stricter.
* tests/cond35.test: Quote literal dots in grep regexp.  Add extra
debugging output.  Minor cosmetic changes.
* tests/cond36.test: Likewise.
* tests/cond41.test: Do not simply check that aclocal fails, but
also grep its stderr for the expected error message.
* tests/cond43.test: Likewise (but the program being automake).
Few cosmetic changes.
* tests/condd.test: Avoid the requirement of a working C compiler.
* tests/condman3.test: Extend by using more man pages and more
manpage sections.
* tests/condman.test: Removed, it's completely superseded by
`condman3.test'.
* tests/Makefile.am (TESTS): Update.

13 years agolex tests: avoid spurious failures when LEXLIB isn't found
Stefano Lattarini [Wed, 25 May 2011 17:05:15 +0000 (19:05 +0200)]
lex tests: avoid spurious failures when LEXLIB isn't found

The AC_PROG_LEX Autoconf macro does not diagnose a failure to find
the "lex library" expected to provide a `yywrap' function (function
which is required to link most lex-generated programs).  On the
contrary, when all the link attempts (i.e., with `-ll' and `-lfl')
fail, configure declares that no lex library is needed, and simply
proceeds with the configuration process -- only for the build to
possibly fail later, at make time.

This behaviour might be (partly) intended; the Autoconf manual
reads:
 ``You are encouraged to use Flex in your sources, since it is
   both more pleasant to use than plain Lex and the C source
   it produces is portable.  In order to ensure portability,
   however, you must either provide a function `yywrap' or, if
   you don't use it (e.g., your scanner has no `#include'-like
   feature), simply include a `%noyywrap' statement in the
   scanner's source.''

This AC_PROG_LEX behaviour is causing some spurious failures of the
Automake testsuite in environments which lack a proper library
providing `yywrap' (this happens for example in Linux->MinGW cross
compilations).  But at this point is clear that a proper workaround
is to simply provide a fall-back implementation of `yywrap' in our
lexers.

* tests/cond35.test: Provide a dummy `yywrap' function.
* tests/lex3.test: Likewise.
* tests/lexvpath.test: Likewise.
* tests/silent-many-gcc.test: Likewise.
* tests/silent-many-generic.test: Likewise.
* tests/silent-lex-gcc.test: Likewise, and a dummy `main' too.
* tests/silent-lex-generic.test: Likewise.
* tests/lex-lib.test: New test.
* tests/lex-libobj.test: New test.
* tests/lex-nowrap.test: New test.
* tests/Makefile.am (TESTS): Update.
* THANKS: Update.

Thanks to Russ Allbery for the suggestion.

13 years agotestsuite: require C++ compiler explicitly in tests needing it
Stefano Lattarini [Wed, 25 May 2011 14:45:30 +0000 (16:45 +0200)]
testsuite: require C++ compiler explicitly in tests needing it

The list of the test scripts which needed a C++ compiler but
failed to require it explicitly has been found by running:
  $ tests=`grep -lE 'CXX|\.c(c|\+\+|xx|pp)' *.test | tr '\n' ' '`
  $ am_explicit_skips=yes CXX=false make check TESTS="$tests"
and looking for tests that reported FAIL instead of SKIP.
After the present change, all those previously failing tests
either pass or get correctly skipped.

* tests/subobj9.test: Require `c++', instead of explicitly
skipping on configure failure.  Add excerpts and/or details from
the original bug report that prompted this tests to be written.
* tests/silentcxx-gcc.test: Require g++.
* tests/silentcxx.test: Require c++.
* tests/suffix3.test: Likewise.

13 years agotestsuite: don't require g++ where any C++ compiler is enough
Stefano Lattarini [Wed, 25 May 2011 14:15:27 +0000 (16:15 +0200)]
testsuite: don't require g++ where any C++ compiler is enough

* tests/specflg10.test ($required): Use `c++', not g++.
* tests/silent-many-generic.test: Likewise.

13 years agotestsuite: require C compiler explicitly in tests needing it
Stefano Lattarini [Mon, 23 May 2011 19:53:44 +0000 (21:53 +0200)]
testsuite: require C compiler explicitly in tests needing it

The list of the test scripts which needed a C compiler but
failed to require it explicitly has been found by running:
  $ am_explicit_skips=yes CC=false make check
and looking for tests that reported FAIL instead of SKIP.
After the present change, all those previously failing tests
either pass or get correctly skipped.

* tests/aclocal4.test ($required): Add `cc'.
* tests/ansi.test: Likewise.
* tests/ansi10.test: Likewise.
* tests/ansi3.test: Likewise.
* tests/ansi3b.test: Likewise.
* tests/ansi5.test: Likewise.
* tests/ansi6.test: Likewise.
* tests/ansi7.test: Likewise.
* tests/ansi9.test: Likewise.
* tests/backcompat6.test: Likewise.
* tests/canon7.test: Likewise.
* tests/check5.test: Likewise.
* tests/check7.test: Likewise.
* tests/check8.test: Likewise.
* tests/cond4.test: Likewise.
* tests/cond11.test: Likewise.
* tests/cond16.test: Likewise.
* tests/cond18.test: Likewise.
* tests/cond19.test: Likewise.
* tests/cond22.test: Likewise.
* tests/cond30.test: Likewise.
* tests/cond31.test: Likewise.
* tests/cond32.test: Likewise.
* tests/cond34.test: Likewise.
* tests/cond35.test: Likewise.
* tests/cond39.test: Likewise.
* tests/condd.test: Likewise.
* tests/cygnus-dependency-tracking.test: Likewise.
* tests/depcomp2.test: Likewise.
* tests/depcomp6.test: Likewise.
* tests/depcomp7.test: Likewise.
* tests/depcomp8a.test: Likewise.
* tests/depcomp8b.test: Likewise.
* tests/depdist.test: Likewise.
* tests/depend2.test: Likewise.
* tests/depend5.test: Likewise.
* tests/distcleancheck.test: Likewise.
* tests/distname.test: Likewise.
* tests/exeext.test: Likewise.
* tests/exeext4.test: Likewise.
* tests/extradep.test: Likewise.
* tests/extradep2.test: Likewise.
* tests/gnits2.test: Likewise.
* tests/gnits3.test: Likewise.
* tests/instdir-ltlib.test: Likewise.
* tests/instdir-prog.test: Likewise.
* tests/instfail.test: Likewise.
* tests/instfail-libtool.test: Likewise.
* tests/lex3.test: Likewise.
* tests/lex5.test: Likewise.
* tests/lexvpath.test: Likewise.
* tests/lex-subobj-nodep.test: Likewise.
* tests/lflags.test: Likewise.
* tests/libobj-basic.test: Likewise.
* tests/libobj2.test: Likewise.
* tests/libobj7.test: Likewise.
* tests/libobj10.test: Likewise.
* tests/libobj16a.test: Likewise.
* tests/libobj16b.test: Likewise.
* tests/libobj17.test: Likewise.
* tests/libobj19.test: Likewise.
* tests/libtool2.test: Likewise.
* tests/libtool3.test: Likewise.
* tests/libtool7.test: Likewise.
* tests/libtool9.test: Likewise.
* tests/libtoo10.test: Likewise.
* tests/libtoo11.test: Likewise.
* tests/ltcond.test: Likewise.
* tests/ltcond2.test: Likewise.
* tests/ltconv.test: Likewise.
* tests/ltinit.test: Likewise.
* tests/ltlibsrc.test: Likewise.
* tests/ltorder.test: Likewise.
* tests/nobase.test: Likewise.
* tests/nobase-libtool.test: Likewise.
* tests/mmodely.test: Likewise.
* tests/parallel-tests5.test: Likewise.
* tests/parallel-tests-suffix-prog.test: Likewise.
* tests/parallel-tests-ext-driver-prog.test: Likewise.
* tests/posixsubst-ldadd.test: Likewise.
* tests/posixsubst-libraries.test: Likewise.
* tests/posixsubst-ltlibraries.test: Likewise.
* tests/posixsubst-programs.test: Likewise.
* tests/posixsubst-sources.test: Likewise.
* tests/pr87.test: Likewise.
* tests/pr204.test: Likewise.
* tests/pr224.test: Likewise.
* tests/pr300-lib.test: Likewise.
* tests/pr300-ltlib.test: Likewise.
* tests/pr300-prog.test: Likewise.
* tests/pr401.test: Likewise.
* tests/pr401b.test: Likewise.
* tests/pr401c.test: Likewise.
* tests/remake-gnulib-add-acsubst.test: Likewise.
* tests/remake-gnulib-add-header.test: Likewise.
* tests/regex.test: Likewise.
* tests/repeated-options.test: Likewise.
* tests/silent.test: Likewise.
* tests/silent3.test: Likewise.
* tests/silent9.test: Likewise.
* tests/silent-lex-generic.test: Likewise.
* tests/silent-many-generic.test: Likewise.
* tests/silent-yacc-generic.test: Likewise.
* tests/specflg7.test: Likewise.
* tests/specflg9.test: Likewise.
* tests/specflg10.test: Likewise.
* tests/stdinc.test: Likewise.
* tests/strip.test: Likewise.
* tests/strip2.test: Likewise.
* tests/strip3.test: Likewise.
* tests/subdirbuiltsources.test: Likewise.
* tests/subobj3.test: Likewise.
* tests/subobj6.test: Likewise.
* tests/subobj11a.test: Likewise.
* tests/subpkg.test: Likewise.
* tests/subst2.test: Likewise.
* tests/subst3.test: Likewise.
* tests/substref.test: Likewise.
* tests/substre2.test: Likewise.
* tests/suffix5.test: Likewise.
* tests/suffix8.test: Likewise.
* tests/suffix10.test: Likewise.
* tests/suffix11.test: Likewise.
* tests/suffix12.test: Likewise.
* tests/suffix13.test: Likewise.
* tests/target-cflags.test: Likewise.
* tests/transform.test: Likewise.
* tests/transform2.test: Likewise.
* tests/yacc-basic.test: Likewise.
* tests/yacc-d-basic.test: Likewise.
* tests/yacc-clean.test: Likewise.
* tests/yacc-dist-nobuild.test: Likewise.
* tests/yacc-nodist.test: Likewise.
* tests/yaccvpath.test: Likewise.
* tests/yacc-d-vpath.test: Likewise.
* tests/yacc4.test: Likewise.
* tests/yacc7.test: Likewise.
* tests/yacc8.test: Likewise.
* tests/yaccdry.test: Likewise.
* tests/yflags-cmdline-override.test: Likewise.
* tests/yflags-force-override.test: Likewise.
* tests/python-virtualenv.test: Likewise.  Also, improve skip
messages.
* tests/subobj5.test ($required): Add `cc'.
(Makefile.am): Set `AUTOMAKE_OPTIONS' to `subdir-objects', and
add new checking rules `test-build' and `test-distdir'.
Extend the test by building and examining the distdir, the
program, and the object files.
* tests/postproc.test ($required): Add `cc'.
Avoid the explicit `|| Exit $?' after call to configure, which
is now either redundant (e.g., when `am_explicit_skips' is
unset), or counter-productive (e.g., when `am_explicit_skips'
is set to "yes").
* tests/pr243.test: Likewise.
* tests/pr266.test: Likewise.
* tests/pr220.test: Simplify so that it doesn't require a
C compiler anymore.
* tests/subdir5.test: Likewise.
* tests/subdir8.test: Likewise.
* tests/lflags.test: Likewise.
* tests/yflags.test: Likewise.
* tests/yflags-force-conditional.test: Likewise.
* tests/lflags2.test: Simplify so that it doesn't require a
C++ compiler anymore.
* tests/yflags2.test: Likewise.
* tests/autohdrdry.test (configure.in): Remove unneeded call
to `AC_PROG_CC'.
* tests/pr287.test: Likewise.
* tests/check6.test: Likewise.
* tests/cond21.test: Likewise, plus some cosmetic adjustments.
* tests/upc.test: Skip if configure fails with status `77'.
* tests/upc3.test: Likewise.
* tests/vala4.test: Likewise.
* tests/nostdinc.test: Likewise.  Also, make grepping checks
on `Makefile.in'.  Update heading comments.
* tests/compile5.test: When we must skip, skip explicitly,
and with a meaningful message.
* tests/instspc-tests.sh: When running in "test-build" or
"test-install" mode (as determined by the value of variable
`$instspc_action'), require `cc'.  Adjust comments.
* tests/depmod-tests.sh: Likewise.

13 years agotestsuite: allow user to ask for stricter skip semantics
Stefano Lattarini [Mon, 23 May 2011 13:51:31 +0000 (15:51 +0200)]
testsuite: allow user to ask for stricter skip semantics

Given how the Automake testsuite is currently structured, if a
command in a test script fails with status `77', the global test
outcome is considered a SKIP, because the value of `77' for `$?'
is passed to the exit trap.  Indeed, this happens in practice, as
an autoconf-generated configure script can exit with status `77'
if it fails to find, e.g., a required compiler.

While this behaviour is quite useful for avoiding spurious test
failures in the wild, it can also sometimes prevent the Automake
developers to easily see and declare the requirements of their
tests.

This change introduces a new variable `am_explicit_skips', meant to
be user-overridable, and which, when set to a "true" value (i.e.,
`yes' or `1'), require a direct call to `Exit 77' in order to make
the test outcome be considered a SKIP.

* tests/defs.in ($am__test_skipped): New variable, initialized
to `no'.
(Exit): Set `$am__test_skipped' to `yes' if passed an exit status
of 77.
(trap '...' 0): When `$am_explicit_skips' is set to a "true" value,
reset an exit status of `77' to `78' if $am__test_skipped is not
set to `yes'.
* tests/self-check-exit.test: Adjust: unset `am_explicit_skips'.
* tests/self-check-explicit-skips.test: New test.
* tests/Makefile.am (TESTS): Update.

13 years agotestsuite: user can force skipping of compiler-requiring tests
Stefano Lattarini [Tue, 24 May 2011 08:42:27 +0000 (10:42 +0200)]
testsuite: user can force skipping of compiler-requiring tests

* tests/defs (cc, c++, fortran, fortran77): Skip the test if
the relevant compiler is disabled by having the corresponding
variable (CC, CXX, FC and F77, respectively) set to "false".
(yacc): For consistency, skip the test when the YACC variable
is set to "false", not when it's set to "no".  Since we are at
it, fix the skip message to be shorter and more consistent.

13 years agotests: tweak and improve tests on "Simple Tests" driver
Stefano Lattarini [Mon, 23 May 2011 12:26:58 +0000 (14:26 +0200)]
tests: tweak and improve tests on "Simple Tests" driver

* tests/check.test: Make grepping of generated Makefile.in
slightly stricter.  Add trailing `:' command.
* tests/check2.test: Ensure verbose printing of captured make
output.  Make grepping of generated Makefile.in and of make
output stricter.  Add trailing `:' command.
* tests/check3.test: Likewise.  Also, fix minor error in a
comment.
* tests/check4.test: Stricter grepping of make output.  Use
`using_gmake' subroutine instead of copy&paste from defs.in
to determine whether $MAKE is GNU make.  Add a trailing `:'
command.
* tests/check6.test: Add trailing `:' command.
* tests/check10.test: Prefer redirection `>' over `>>' when
creating new files.  Ensure verbose printing of captured make
output.
* tests/check11.test: Likewise.
* tests/check7.test: Likewise.  Make grepping of make output
stricter.  Prefer "make distcheck" over a simple "make check"
in a VPATH build. Add trailing `:' command.
(a.c): Format following GNU coding standards.
(Makefile.am): Explicitly declare target `print-xfail-tests'
as PHONY.  Distribute $(check_SCRIPTS), for distcheck.
* tests/parallel-tests.test: Prefer redirection `>' over `>>'
when creating new files.
* tests/parallel-tests2.test: Likewise.
* tests/parallel-tests9.test: Likewise.
* tests/parallel-tests8.test: Likewise.  Make grepping of
automake stderr stricter.
* tests/check8.test: Likewise.  Make formatting of created
*.c files more consistent with GNU Coding Standards.
* tests/check-subst-prog.test (configure.in): Break overly
long lines.
(pass-prog.c, xfail-prog.c): Format them more consistently
with GNU Coding Standards.
* tests/parallel-tests3.test: Use a shorter and clearer skip
message.  Use the `unindent' subroutine to improve readability
of indented loop bodies.  Simplify the hack used to please
maintainer-check.
* tests/parallel-tests5.test: Make formatting of created *.c
files more consistent with GNU Coding Standards.  Use the
`unindent' subroutine to improve readability of indented loop
bodies.
* tests/parallel-tests10.test: Add blank line, for clarity.
Prefer trailing `:' over trailing `Exit 0', for consistency
with other tests.

13 years agotests/README: fix example about `make -e' usage
Ralf Wildenhues [Sun, 22 May 2011 17:02:27 +0000 (19:02 +0200)]
tests/README: fix example about `make -e' usage

* tests/README (Section "Writing test cases" subsection "Do"): When
some variable is never initialized in the Makefile, `-e' is not
necessary in order to override it.  DESTDIR is such a variable: we
ensure that we do not ever initialize it.  And as such, it is quite
portable to use:
  $ make DESTDIR=/foo/bar install
and in fact, quite widely used.
So our example about when `make -e' is required, which references
the `DESTDIR' variable, is poorly chosen, if not downright wrong.
Rewrite it to use `prefix' as the overridden variable instead.

13 years agotests/README: small consistency improvement
Stefano Lattarini [Sat, 21 May 2011 09:03:00 +0000 (11:03 +0200)]
tests/README: small consistency improvement

* tests/README (Section "User interface" subsection "Running the
tests"): In examples, be more consistent w.r.t. the differences
between GNU and non-GNU make.

13 years agotests/README: suggest how to run tests in cross-compile mode
Stefano Lattarini [Sat, 21 May 2011 08:59:53 +0000 (10:59 +0200)]
tests/README: suggest how to run tests in cross-compile mode

* tests/README (Section "User interface" subsection "Running the
tests"): Briefly explain how to override 'host_alias' at runtime
to force the use of cross-compilers by the testsuite.  Give an
example.

13 years agotest defs: better detection of cross-compile mode
Stefano Lattarini [Sat, 21 May 2011 07:12:06 +0000 (09:12 +0200)]
test defs: better detection of cross-compile mode

* configure.ac (AC_CANONICAL_HOST): New, probably not strictly
necessary, but useful to complements AC_CANONICAL_BUILD.
* tests/defs-static.in ($host_alias, $build_alias): New variables,
exported, user-overridable, defaults AC_SUBST'd by configure.
* tests/defs (cross_compiling): Consider cross-compilation mode
active only if `$host_alias' is not empty *and* it differs from
`$build_alias'.  This is more faithful to the way configure
determines whether it is cross-compiling, but still allows the
user to easily force cross-compilation by overriding `host_alias'
and `build_alias' at runtime.

From a report by Ralf Wildenhues.

13 years agoMerge branch 'master' into testsuite-work
Stefano Lattarini [Fri, 20 May 2011 22:08:02 +0000 (00:08 +0200)]
Merge branch 'master' into testsuite-work

* master:
  testsuite: avoid re-running few tests with 'parallel-tests' option
  testsuite: allow tests to avoid the use of 'parallel-tests' option
  tests/README: update obsoleted advice

13 years agoMerge branch 'maint'
Stefano Lattarini [Fri, 20 May 2011 21:33:09 +0000 (23:33 +0200)]
Merge branch 'maint'

* maint:
  testsuite: avoid re-running few tests with 'parallel-tests' option
  testsuite: allow tests to avoid the use of 'parallel-tests' option
  tests/README: update obsoleted advice

13 years agotestsuite: avoid re-running few tests with 'parallel-tests' option
Stefano Lattarini [Fri, 20 May 2011 21:18:39 +0000 (23:18 +0200)]
testsuite: avoid re-running few tests with 'parallel-tests' option

Some tests in our testsuite use the 'simple-tests' driver only
marginally, or simply as a mean to conveniently check unrelated
invariants.  It makes little sense to force these tests to also
run with the 'parallel-tests' Automake option active, as doing so
offers no real gain in coverage, while often causing a measurable
overhead in execution time (for an already too-slow testsuite).

* tests/pr401.test (parallel_tests): Define to "no", to prevent
the generation of a sibling test script using the 'parallel-tests'
driver.
* tests/pr401b.test: Likewise.
* tests/pr401c.test: Likewise.

13 years agotestsuite: allow tests to avoid the use of 'parallel-tests' option
Stefano Lattarini [Fri, 20 May 2011 20:31:40 +0000 (22:31 +0200)]
testsuite: allow tests to avoid the use of 'parallel-tests' option

* tests/gen-parallel-tests: Do not generate "siblings" for tests
that explicitly define the `parallel_tests', whether to "yes" or
to any other value.  Extend heading comments to give a rationale
for this behaviour.
* tests/README: Update.

13 years agotests/README: update obsoleted advice
Stefano Lattarini [Fri, 20 May 2011 20:15:32 +0000 (22:15 +0200)]
tests/README: update obsoleted advice

* tests/README (Section "Writing test cases" subsection "Do"):
Do not suggest to use the `*-p.test' pattern for the names of
hand-written tests which use the `parallel-tests' Automake option.
Not only is this not respected by the existing tests, but it is
more likely to cause conflicts with auto-generated tests.
So, suggest to *avoid* using the `*-p.test' pattern in names
of hand-written tests instead.
(Section "Writing test cases" subsection "Do not"):  When
suggesting not to override Makefile variables using command
line arguments, do not use the badly outdated variables `U'
and 'ANSI2KNR' in the example; instead, use the more common
and typical `DESTDIR'.

13 years agofix typos in latest ChangeLog entry
Stefano Lattarini [Thu, 19 May 2011 20:50:35 +0000 (22:50 +0200)]
fix typos in latest ChangeLog entry

13 years agotests: avoid spurious failures in cross-compile mode
Stefano Lattarini [Sun, 15 May 2011 14:33:01 +0000 (16:33 +0200)]
tests: avoid spurious failures in cross-compile mode

* tests/depcomp2.test: Ensure verbose printing of captured stderr
from configure.
* tests/ansi3.test ($required): Add 'native', as the ansi2knr
functionality is not meant to work with a cross-compiler.
* tests/ansi3b.test: Likewise.
* tests/ansi5.test: Likewise.
* tests/ansi6.test: Likewise.
* tests/ansi7.test: Likewise.
* tests/ansi8.test: Likewise.
* tests/ansi10.test: Likewise.
* tests/subobj3.test: Likewise.
* tests/check8.test ($required): Add 'native', as this test is
not meant to work with a cross-compiler.
* tests/gnits2.test: Likewise.
* tests/ltconv.test: Likewise.
* tests/ltcond2.test: Likewise.
* tests/parallel-tests5.test: Likewise.
* tests/specflg7.test: Likewise.
* tests/specflg8.test: Likewise.
* tests/check5.test: Likewise.  Also, ensure verbose printing of
captured make output, and thrown in few cosmetic and consistency
improvements.
* tests/ltinit.test: Be laxer in grepping configure output, to
avoid spurious failures on systems which lack POSIX dynamic
linking (e.g., MinGW), or when cross-compiling for such systems.
* tests/ansi9.test: Skip parts of the test that would require a
native compiler.
* tests/backcompat6.test: Likewise.
* tests/depcomp8a.test: Likewise.
* tests/pr401.test: Likewise.
* tests/pr401b.test: Likewise.
* tests/pr401c.test: Likewise.
* tests/target-cflags.test: Likewise.
* tests/lex3.test: Likewise.
* tests/yacc-basic.test: Likewise.
* tests/autohdr4.test: Likewise.  Also, extend the test to ensure
more coverage in cross-compile mode.
* tests/canon7.test: Likewise.
* tests/check9.test: Removed, split into two new tests ...
* tests/check-subst-prog.test: ... this one, which requires a
native C compiler ...
* tests/check-subst.test: ... and this one, which doesn't require
any compiler.
* tests/parallel-tests4.test: Removed, split into two new tests ...
* tests/parallel-tests-suffix-prog.test: ... this one, which
requires a native C compiler ...
* tests/parallel-tests-suffix.test: ... and this one, which
doesn't require any compiler.
* tests/parallel-tests7.test: Removed, split into two new tests ...
* tests/parallel-tests-ext-driver-prog.test: ... this one, which
requires a native C compiler ...
* tests/parallel-tests-ext-driver.test: ... and this one, which
doesn't require any compiler.
* tests/Makefile.am (TESTS): Update.

13 years agoMerge branch 'master' into testsuite-work
Stefano Lattarini [Thu, 19 May 2011 20:07:48 +0000 (22:07 +0200)]
Merge branch 'master' into testsuite-work

* master:
  test defs: rename requirement 'non-cross' -> 'native'

13 years agoMerge branch 'maint'
Stefano Lattarini [Thu, 19 May 2011 19:53:02 +0000 (21:53 +0200)]
Merge branch 'maint'

* maint:
  test defs: rename requirement 'non-cross' -> 'native'

13 years agotest defs: rename requirement 'non-cross' -> 'native'
Stefano Lattarini [Thu, 19 May 2011 19:48:27 +0000 (21:48 +0200)]
test defs: rename requirement 'non-cross' -> 'native'

* tests/defs.in (non-cross): Rename requirement ...
(native): ... to this, which is clearer and fits the
existing lingo better.

Suggestion by Ralf Wildenhues.

13 years agoMerge branch 'master' into testsuite-work
Stefano Lattarini [Sun, 15 May 2011 13:38:14 +0000 (15:38 +0200)]
Merge branch 'master' into testsuite-work

* master:
  testsuite: be more cross-compile friendly
  tests: fix portability issues in 'repeated-options.test'

13 years agoMerge branch 'maint'
Stefano Lattarini [Sun, 15 May 2011 13:35:27 +0000 (15:35 +0200)]
Merge branch 'maint'

* maint:
  testsuite: be more cross-compile friendly

13 years agotestsuite: be more cross-compile friendly
Stefano Lattarini [Sun, 15 May 2011 12:30:43 +0000 (14:30 +0200)]
testsuite: be more cross-compile friendly

* tests/defs.in (cross_compiling): New subroutine.
(am__tool_prefix): New internal variable.
(gcc, g++, gcj): Force the use of the correct "tool prefix"
when cross compiling.
(gfortran, g77, non-cross): New requirements.

13 years agoMerge branch 'test-repeated-options-portability-fix'
Stefano Lattarini [Sun, 15 May 2011 12:41:57 +0000 (14:41 +0200)]
Merge branch 'test-repeated-options-portability-fix'

13 years agotests: fix portability issues in 'repeated-options.test'
Stefano Lattarini [Sun, 15 May 2011 12:41:35 +0000 (14:41 +0200)]
tests: fix portability issues in 'repeated-options.test'

* tests/repeated-options.test: Do not assume that object files
have `.o' suffix and executables have no default suffix; instead,
use `$(OBJEXT)' and `$(EXEEXT)'.

13 years agoMerge branch 'master' into testsuite-work
Stefano Lattarini [Sat, 7 May 2011 13:40:54 +0000 (15:40 +0200)]
Merge branch 'master' into testsuite-work

* master:
  tests: fix spurious failure of extradep.test on FreeBSD

13 years agoMerge branch 'maint'
Stefano Lattarini [Sat, 7 May 2011 13:39:37 +0000 (15:39 +0200)]
Merge branch 'maint'

* maint:
  tests: fix spurious failure of txinfo21.test on FreeBSD

13 years agoMerge branch 'maint' into testsuite-work
Stefano Lattarini [Sat, 7 May 2011 13:38:56 +0000 (15:38 +0200)]
Merge branch 'maint' into testsuite-work

* maint:
  tests: fix spurious failure of txinfo21.test on FreeBSD

13 years agoMerge branch 'test-silentcxx-gcc-fix' into testsuite-work
Stefano Lattarini [Sat, 7 May 2011 13:35:35 +0000 (15:35 +0200)]
Merge branch 'test-silentcxx-gcc-fix' into testsuite-work

13 years agotests: fix syntax error in 'silentcxx-gcc.test'
Stefano Lattarini [Sat, 7 May 2011 12:18:47 +0000 (14:18 +0200)]
tests: fix syntax error in 'silentcxx-gcc.test'

* tests/silentcxx-gcc.test (for config_args in ...): Remove extra
trailing backslash from the list of looped-on items.

13 years agoMerge branch 'test-extradep-fix'
Stefano Lattarini [Sat, 7 May 2011 13:23:57 +0000 (15:23 +0200)]
Merge branch 'test-extradep-fix'

13 years agotypofix in ChangeLog
Stefano Lattarini [Sat, 7 May 2011 13:23:28 +0000 (15:23 +0200)]
typofix in ChangeLog

13 years agoMerge branch 'test-extradep-fix'
Stefano Lattarini [Sat, 7 May 2011 13:10:14 +0000 (15:10 +0200)]
Merge branch 'test-extradep-fix'

13 years agotests: fix spurious failure of extradep.test on FreeBSD
Stefano Lattarini [Sat, 7 May 2011 10:58:51 +0000 (12:58 +0200)]
tests: fix spurious failure of extradep.test on FreeBSD

* tests/extradep.test: When using `ls -t' to determine whether a
file has been updated, make sure to use as reference a file whose
timestamp is expected to be *strictly* older that that of the file
being checked.  This is required because at least FreeBSD `ls' do
not sort files with the same timestamp in alphabetical order when
using the `-t' option.
* tests/extradep2.test: Likewise.

13 years agotests: fix spurious failure of txinfo21.test on FreeBSD
Stefano Lattarini [Sat, 7 May 2011 12:34:39 +0000 (14:34 +0200)]
tests: fix spurious failure of txinfo21.test on FreeBSD

* tests/txinfo21.test: Use the `is_newest' subroutine instead of
the `ls -t' hack to to determine whether a file has been updated.
This is required because at least FreeBSD `ls' do not sort files
with the same timestamp in alphabetical order when using the `-t'
option.

13 years agoMerge branch 'fix-depmod-tests-generation' into testsuite-work
Stefano Lattarini [Sat, 7 May 2011 09:50:02 +0000 (11:50 +0200)]
Merge branch 'fix-depmod-tests-generation' into testsuite-work

13 years agodepmod tests: fix bug in depmodes list extraction
Stefano Lattarini [Sat, 7 May 2011 09:49:40 +0000 (11:49 +0200)]
depmod tests: fix bug in depmodes list extraction

* tests/depmod-tests.sh (get_depmodes): Use `echo $all_depmodes'
instead of `... | tr "$nl" "$sp"' to strip extra whitespaces and
newlines from `$all_depmodes'.  The latter idiom with `tr' is
wrong since it gets run before `./defs' is sourced, and thus `$nl'
and `$sp' are undefined.
Bug revealad by FreeBSD tr(1) implementation, which doesn't accept
empty strings as arguments.

13 years agotests: add excerpts from few original bug reports, for clarity.
Stefano Lattarini [Wed, 4 May 2011 21:36:25 +0000 (23:36 +0200)]
tests: add excerpts from few original bug reports, for clarity.

* tests/comment6.test: Add excerpts and/or details from the
original bug report that prompted this tests to be added and/or
extended.
* tests/dejagnu4.test: Likewise.
* tests/installdir.test: Likewise.

13 years agoMerge branch 'master' into testsuite-work
Stefano Lattarini [Wed, 4 May 2011 21:09:54 +0000 (23:09 +0200)]
Merge branch 'master' into testsuite-work

13 years agoChangeLog: some typofixes
Stefano Lattarini [Wed, 4 May 2011 21:06:16 +0000 (23:06 +0200)]
ChangeLog: some typofixes

13 years agoMerge branch 'maint'
Stefano Lattarini [Wed, 4 May 2011 20:50:59 +0000 (22:50 +0200)]
Merge branch 'maint'

13 years agoChangeLog: various typofixes (and related reformatting)
Stefano Lattarini [Wed, 4 May 2011 20:42:19 +0000 (22:42 +0200)]
ChangeLog: various typofixes (and related reformatting)

13 years agoMerge branch 'maint'
Stefano Lattarini [Wed, 4 May 2011 20:21:53 +0000 (22:21 +0200)]
Merge branch 'maint'

With some edits to `tests/defs' and `ChangeLog', to reflect the
fact that in master we already had a `yacc' requirement, and no
more had an explicitly-recognized `bison' requirement.

13 years agotests defs: allow requirements for compilers (mostly dummy)
Stefano Lattarini [Wed, 4 May 2011 16:42:11 +0000 (18:42 +0200)]
tests defs: allow requirements for compilers (mostly dummy)

Most of the new requirements that are now accepted in `$required'
as consequence of this patch are still dummy.  They are planned
to be implemented only in master (or in some derived branch), but
having them here (even just as no-op) will allow for an easier
integration/backporting of potential new testcases.

* tests/defs.in (cc, c++, fortran, fortran77): New requirements,
still dummy.
(flex): New requirement, picking LEX for configure.
(lex): New requirement, alias for `flex'.  A more appropriate
implementation, looking for a generic `lex' program, will follow
in the future.
(yacc): New requirement, alias for `bison'.  A more appropriate
implementation, looking for a generic `yacc' program, will follow
in the future.

13 years agotests: 'silent-many-gcc.test' improved and made more robust
Stefano Lattarini [Wed, 4 May 2011 09:58:21 +0000 (11:58 +0200)]
tests: 'silent-many-gcc.test' improved and made more robust

* tests/silent-many-gcc.test:  Also force "fast" gcc depmode
for C++ compilations.  Add sanity checks verifying that the
cache variables we force are really used by configure.  Fix
typo in comments.

13 years agotests: drop useless requirements of 'g++'
Stefano Lattarini [Wed, 4 May 2011 09:48:19 +0000 (11:48 +0200)]
tests: drop useless requirements of 'g++'

* tests/lflags2.test ($required): Remove 'g++', as any working
C++ compiler should be enough, and ./configure will cause the
test to skip if no working C++ compiler is found.
* tests/yflags2.test: Likewise.
* tests/subobj9.test: Likewise.
* tests/silentcxx.test: Likewise.  Also, do not force depmodes
that could cause non-GNU C++ compilers to fail.
* tests/silentcxx-gcc.test: New test, like `silentcxx.test',
but forcing "fast" gcc depmode (and thus requiring the GNU C++
compiler).
* tests/specflg10.test: Add proper "fixme" comment telling that
we should make this test work with a generic C++ compiler.
* tests/Makefile.am (TESTS): Update.