Stefano Lattarini [Sun, 5 Jun 2011 18:58:21 +0000 (20:58 +0200)]
self tests: check that `$me' can be overridden
* tests/self-check-me.test: Check that `$me' can be overridden
before sourcing ./defs, with or without sourcing ./defs-static
beforehand, and that this override is honored. Update heading
comments.
Stefano Lattarini [Sun, 5 Jun 2011 10:16:48 +0000 (12:16 +0200)]
Revert "tests: use `$SHELL' to run the shell scripts from `lib/'"
This reverts commit
f977d00e0a3ab9af3555bb3bc8ea78726cb7a143.
See <http://lists.gnu.org/archive/html/automake-patches/2011-06/msg00018.html>
for the reasons behind this revert.
Stefano Lattarini [Thu, 2 Jun 2011 15:54:25 +0000 (17:54 +0200)]
self tests: fix another spurious failure
Our ad-hoc usage of `tests/defs' in the testsuite's self tests
stopped working properly when we made the test scripts re-execute
themselves with the configure-time $SHELL. Fix this.
Fixes a regression introduced by commit 'v1.11-874-g1321be7'.
* tests/self-check-exit.test: Export `AM_TESTS_REEXEC' to "no"
before running the self tests. This fixes a spurious failure
present only when the test was run by hand.
Stefano Lattarini [Thu, 2 Jun 2011 15:05:51 +0000 (17:05 +0200)]
tests: use `$SHELL' to run the shell scripts from `lib/'
This should offer greater testsuite coverage for those developers
that override CONFIG_SHELL at configure time in order to test more
shells on a single system, instead of just the default `/bin/sh'.
This change also fixes few spurious failures in tests using the
`xsi-shell' requirement, where inconsistencies could crop up if
the shell probed for XSI features (which, by default, is $SHELL)
was not the same shell later used to run the scripts using those
features (which was hard-coded to `/bin/sh'). Such failures have
already occurred in practice, for examples on Solaris systems
which had also GNU Bash installed.
* tests/ar-lib.test: Run the `ar-lib' script with `$SHELL', rather
than directly with `./ar-lib', which would make run unconditionally
with `/bin/sh'.
* tests/compile.test: Likewise, but for the `compile' script.
* tests/compile2.test: Likewise.
* tests/compile3.test: Likewise.
* tests/compile4.test: Likewise.
* tests/compile5.test: Likewise.
* tests/compile6.test: Likewise.
* tests/instsh2.test: Likewise, but for the `install' script.
* tests/instsh3.test: Likewise.
* tests/mkinst3.test: Likewise, but for the `mkinstalldirs' script.
* tests/missing.test: Likewise, but for the `missing' script.
* tests/missing2.test: Likewise.
* tests/missing3.test: Likewise.
* tests/missing5.test: Likewise.
Stefano Lattarini [Thu, 2 Jun 2011 13:35:03 +0000 (15:35 +0200)]
silent-rules tests: fix spurious failures with Sun Studio C++
* tests/silentcxx.test: The C++ compiler from Sun Studio is named
`CC'. Account for this in our grepping checks on the make output.
Since we are at it, throw in a couple of improvements to comments
and formatting.
* tests/silent-many-generic.test: The C++ compiler from Sun Studio
is named `CC', and this can cause spurious failures in our grepping
of the make output. Work around this by using a wrapper script
around the C++ compiler (generated on the fly), since filtering the
make output proved to be too fragile.
Stefano Lattarini [Thu, 2 Jun 2011 12:40:19 +0000 (14:40 +0200)]
tests: fix spurious failure in backcompat2.test on NetBSD
* tests/backcompat2.test: Add trailing `:' in the body of a `for'
loop, in case the last command there might have an exit status
different 0. This work around `set -e' issues in some BSD shells,
e.g., NetBSD /bin/ksh.
Stefano Lattarini [Thu, 2 Jun 2011 12:14:28 +0000 (14:14 +0200)]
tests: fix spurious failure in autohdr3.test
* tests/autohder3.test (Makefile.am): Let `test' depend on `all',
so that config.h header is truly remade.
Remove now useless call to `$MAKE' when non-GNU make is in use.
Failure introduced by commit `v1.11-895-g5e62b96'.
Stefano Lattarini [Thu, 2 Jun 2011 12:13:18 +0000 (14:13 +0200)]
tests: fix spurious failure of cond29.test on NetBSD
* tests/cond29.test: Skip if we cannot safely limit the maximal
size of used virtual memory to 20K. This fixes a spurious failure
on NetBSD.
Bug introduced in commit `v1.11-885-g908d335'.
Stefano Lattarini [Thu, 2 Jun 2011 10:58:37 +0000 (12:58 +0200)]
maintcheck: fix again few more failures
* tests/cond29.test: To please the `sc_tests_Exit_not_exit'
maintainer check, avoid using `sh -c "exit 0"' where a simple
`sh -c :' will do.
* tests/depmod-data.test: Use creative quoting to avoid
spuriously triggering the `sc_tests_Exit_not_exit' maintainer
check.
Stefano Lattarini [Thu, 2 Jun 2011 10:54:03 +0000 (12:54 +0200)]
Merge branch 'master' into testsuite-work
* master:
maintcheck: fix some more failures
maintcheck: fix some failures, extend some checks
Stefano Lattarini [Thu, 2 Jun 2011 10:47:58 +0000 (12:47 +0200)]
maintcheck: fix some more failures
* tests/instdir-ltlib.test: Use creative quoting to avoid
spuriously triggering the `sc_rm_minus_f' maintainer check.
* tests/instdir-prog.test: Likewise.
* tests/instspc-data.test: Use creative quoting to avoid
spuriously triggering the `sc_tests_Exit_not_exit' maintainer
check.
Stefano Lattarini [Thu, 2 Jun 2011 10:17:40 +0000 (12:17 +0200)]
Merge branch 'maint'
* maint:
maintcheck: fix some failures, extend some checks
Stefano Lattarini [Thu, 2 Jun 2011 10:15:52 +0000 (12:15 +0200)]
maintcheck: fix some failures, extend some checks
* Makefile.am (sc_diff_automake_in_automake): Update, as we
now expect 9 lines, not 8, to be changed from `automake.in'
to `automake'.
(sc_diff_aclocal_in_aclocal): New maintainer check, similar to
the above, and checking that only 10 lines are changed from
`aclocal.in' to `aclocal'.
(syntax_check_rules): Update.
(sc_tests_Exit_not_exit): Exempt self tests `self-check-*.test'
from this check, as they can legitimately use the bare `exit'
builtin in various places.
* doc/automake.texi (Python): Remove stray `@' from the end of
a line. Typo introduced in commit `v1.11-312-g5bf7af6'.
* tests/depcomp8a.test: Pass DISTCHECK_CONFIGURE_FLAGS to make
from the environment rather than from the command line, to
pacify the `sc_tests_overriding_macros_on_cmdline' maintainer
check.
* tests/depcomp8b.test: Likewise.
Stefano Lattarini [Thu, 2 Jun 2011 09:22:32 +0000 (11:22 +0200)]
tests: don't require GNU make where it's not strictly needed
* tests/autohdr3.test: Rewrite to be stricter when make is GNU
make. Drop the requirement of GNU make.
* tests/aclocal5.test: Drop GNU make requirement, it's not truly
needed. Add reference to similar tests `remake-subdir*.test'.
* remake-subdir-gnu.test, remake-subdir-from-subdir.test,
remake-subdir.test, remake-subdir2.test: Add reference to
each other, and to related test `aclocal5.test'.
* tests/aclocal6.test: Drop GNU make requirement, it's not
truly needed.
* tests/confh6.test: Likewise.
* tests/lex3.test: Likewise.
* tests/remake11.test: Likewise.
* tests/subdir5.test: Likewise.
* tests/subdir8.test: Likewise.
* tests/werror2.test: Likewise.
* tests/conff.test: Likewise, and ensure verbose printing of
captured make output.
* tests/lex5.test: Tweak so that GNU make is no more required.
* tests/version7.test: Likewise.
* tests/maken2.test: Add explicative comment for why this test
required GNU make.
* tests/maken4.test: Let it run also with BSD makes supporting
the `.MAKE' special target.
* tests/output6.test: Use proper m4 quoting in configure.in.
Expand make macros with one-character name using `$(x)', not
`$x', for portability. Move checks in the makefiles, rather
than relying on grepping the output from make. Drop the now
unneeded GNU make requirement.
Stefano Lattarini [Wed, 1 Jun 2011 18:23:45 +0000 (20:23 +0200)]
Merge branch 'master' into testsuite-work
* master:
remake: behave better with non-GNU make in subdirectories
Stefano Lattarini [Wed, 1 Jun 2011 17:37:18 +0000 (19:37 +0200)]
Merge branch 'maint'
* maint:
remake: behave better with non-GNU make in subdirectories
Stefano Lattarini [Wed, 1 Jun 2011 16:58:35 +0000 (18:58 +0200)]
Merge branch 'remake-rules-non-GNU-make' into maint
Stefano Lattarini [Wed, 1 Jun 2011 16:49:27 +0000 (18:49 +0200)]
lex tests: fix spurious failures with Solaris lex
* tests/lex-lib.test (foo.l): Avoid empty "rules section", which
can confuse Solaris lex.
* tests/lex-libobj.test (foo.l): Likewise.
Stefano Lattarini [Wed, 1 Jun 2011 15:55:05 +0000 (17:55 +0200)]
lex tests: do not force the use of flex unconditionally
* tests/defs (lex): Act more similarly to what the `yacc'
requirement does, i.e., only force the use of flex if the
$LEX variable is left unset by the user.
(flex): Use `skip_' to skip the test if flex is not found.
Stefano Lattarini [Wed, 1 Jun 2011 15:43:44 +0000 (17:43 +0200)]
lex tests: avoid possible hang; fix and extend
* tests/lex3.test (foo.l:yywrap): Return 1, not 0, to avoid hangs.
Bug introduced in commit 'v1.11-871-geb147a1'.
(Makefile.am): Do not add `@LEXLIB@' to `$(LDADD)', as we define
our own `yywrap' function.
* tests/lex.test (tscan.l): In `yywrap', return 1, not 0, for
consistency with the default flex implementation.
* tests/lex-libobj.test (yywrap.c): Likewise.
* tests/lex-subobj-nodep.test (s1.l): Likewise.
* tests/lexvpath.test (foo.c): Likewise.
* tests/silent-lex-gcc (foo.l): Likewise.
* tests/silent-lex-generic (foo.l): Likewise.
* tests/silent-many-gcc (foo5.l): Likewise.
* tests/silent-many-generic (foo5.l): Likewise.
* tests/lex-lib.test (mu.c): Likewise.
Update heading comments, to refer to ...
* tests/lex-lib-external.test: ... this new test, which checks
that we can get use the `yywrap' function from a system-wide
library, if that's available.
Stefano Lattarini [Wed, 1 Jun 2011 14:45:19 +0000 (16:45 +0200)]
tests: prefer `skip_' over `echo ...; Exit 77'
* tests/self-check-cleanup.test: When the test must be skipped,
use `skip_ REASON' instead of `echo REASON; Exit 77'. Also,
make the skip message shorter and clearer.
Stefano Lattarini [Wed, 1 Jun 2011 14:33:07 +0000 (16:33 +0200)]
tests: fix spurious failures in self tests
Our ad-hoc usage of `tests/defs' in the testsuite's self tests
stopped working properly when we made the test scripts re-execute
themselves with the configure-time $SHELL. Fix this.
Fixes a bug introduced by commit 'v1.11-874-g1321be7'.
* tests/defs: Only check that we can find the client test script
when we must re-execute it.
* tests/self-check-cleanup.test: Export `AM_TESTS_REEXEC' to "no"
before running the self tests.
* tests/self-check-dir.test: Likewise.
* tests/self-check-explicit-skips.test: Likewise.
* tests/self-check-me.test: Likewise.
* tests/self-check-sanity.test: Likewise.
* tests/self-check-reexec.test: New test.
* tests/Makefile.am (TESTS): Update.
Stefano Lattarini [Wed, 1 Jun 2011 10:09:23 +0000 (12:09 +0200)]
tests: prefer ulimit over timeout in test on a memory-hogging bug
* tests/cond29.test: Use 'ulimit' instead of 'timeout' to ensure
that automake does not uses up too much resources. This is really
and improvement because the bug tested by this script wasn't just
a "it hangs" or "it runs too slow" bug, but rather a memory-hogging
bug (due to combinatorial explosion when many Automake conditionals
had to be handled) which could easily crash the whole system, which
is unacceptable. The requirement of a proper and working 'ulimit'
builtin might cause the test to be skipped on more systems, but
that shouldn't be a problem since the bug isn't about a portability
issue, but is rather an automake internal implementation problem.
I've verified that the test as updated by this patch still passes
with automake 1.8.5, automake 1.10.2, and obviously the development
version of automake, and that it fails with automake 1.7.9.
Stefano Lattarini [Tue, 31 May 2011 19:42:16 +0000 (21:42 +0200)]
testsuite: avoid generating `*-p.test' tests, use a wrapper script
The generated `*-p.test' tests had already become just thin layers
around the corresponding test scripts. This change makes the final
step, converting to the use of a generic wrapper script and thus
avoiding the extra test generation (similarly to what is done for
the `*.instspc' and `*.depmod' tests).
* tests/parallel-tests.sh: New file, driver script to run checks
the on the `parallel-tests' semantics by wrapping tests that use
the generic "Simple Tests" driver.
* tests/gen-parallel-tests: Update, mostly to reflect the new
`.ptest' extensions used for tests in $(parallel_tests).
* Makefile.am (TESTS_EXTENSIONS): Add `.ptest'.
(PTEST_LOG_COMPILER): Define, it calls `parallel-tests.sh'.
($(parallel_tests)): Do not really generate `*-p.test' tests
anymore; this is now just a dummy dependency declaration required
in order to have make actually produce expected log files from
the `.ptest.log' suffix rule.
(EXTRA_DIST): Distribute `parallel-tests.sh'.
(MAINTAINERCLEANFILES): Don't remove the `$(parallel_tests)', it
is not necessary anymore.
(generated_tests): Variable definition removed.
(TESTS): Update, by listing `$(parallel_tests)' directly instead
of `$(generated_tests)'
(expected_list_of_tests): Remove `$(generated_tests)'.
(maintainer-check-list-of-tests): No need to explicitly depend on
`$(expected_list_of_tests)' anymore.
Stefano Lattarini [Tue, 31 May 2011 16:07:56 +0000 (18:07 +0200)]
Merge branch 'master' into testsuite-work
* master:
automake, aclocal: honour configure-time AUTOCONF and AUTOM4TE
build: the user can override AUTOM4TE, AUTORECONF and AUTOUPDATE too
tests/README: fix example about `make -e' usage
Stefano Lattarini [Tue, 31 May 2011 16:03:48 +0000 (18:03 +0200)]
Merge branch 'maint'
* maint:
automake, aclocal: honour configure-time AUTOCONF and AUTOM4TE
build: the user can override AUTOM4TE, AUTORECONF and AUTOUPDATE too
tests/README: fix example about `make -e' usage
Stefano Lattarini [Sun, 29 May 2011 08:42:00 +0000 (10:42 +0200)]
automake, aclocal: honour configure-time AUTOCONF and AUTOM4TE
Currently, the Automake's own configure script allow definition
of AUTOCONF and AUTOM4TE, expected to point respectively to an
autoconf and autom4te programs. But while these definitions are
honoured in the Automake's build systems and test suite, they
were *not* honoured in the generated `automake' and `aclocal'
scripts. This behaviour, apart from being wrong in that it does
not allow the user enough freedom in choosing his tools, also
caused inconsistencies in the test suite, brining to spurious
failures.
Problem reported by Graham Reitz on the automake list; see thread:
<http://lists.gnu.org/archive/html/automake/2011-05/msg00022.html>
* automake.in ($traces): Use `@am_AUTOCONF', not simply `autoconf'.
* aclocal.in ($traces): Use `@am_AUTOM4TE', not simply `autom4te'.
* Makefile.am (do_subst): Substitute also `@am_AUTOCONF' and
`@am_AUTOM4TE'.
* NEWS: Update.
* THANKS: Update.
Stefano Lattarini [Sun, 29 May 2011 09:04:08 +0000 (11:04 +0200)]
build: the user can override AUTOM4TE, AUTORECONF and AUTOUPDATE too
Our build system allows the user to override AUTOCONF and AUTOHEADER
at configure time, and honours these overrides in our testsuite.
But it didn't do the same with AUTOM4TE, AUTORECONF and AUTOUPDATE.
This change fixes that inconsistency.
* configure.ac (am_AUTOM4TE, am_AUTOUPDATE, am_AUTORECONF): New
AC_SUBSTitutions. Update comments.
* tests/defs.in ($AUTOUPDATE): Default to `@am_AUTOUPDATE@' now.
($AUTOM4TE): New variable, defaulting to `@am_AUTOM4TE@'.
($AUTORECONF): New variable, defaulting to `@am_AUTORECONF@'.
* doc/Makefile.am ($(srcdir)/amhello-1.0.tar.gz): Update.
Stefano Lattarini [Sun, 29 May 2011 13:24:58 +0000 (15:24 +0200)]
remake: behave better with non-GNU make in subdirectories
Currently, with every decent make program, it is possible to
rebuild out-of-date autotools-generated files with a simple
"make Makefile" -- but for this to work reliably with non-GNU
make implementations, the command must be issued from the
top-level directory. This patch removes such limitation.
* lib/am/configure.am (am--refresh): Depend on `%MAKEFILE%'.
* tests/defs.in (using_gmake): New function, backported from the
`master' branch (and simplified).
* tests/remake-subdir.test: New test.
* tests/remake-subdir2.test: Likewise.
* tests/remake-subdir-gnu.test: Likewise.
* tests/remake-subdir-from-subdir.test: Likewise.
* tests/Makefile.am (TESTS): Update.
Stefano Lattarini [Sun, 29 May 2011 16:13:11 +0000 (18:13 +0200)]
tests: don't require gfortran if any fortran compiler is enough
* tests/silentf90.test ($required): Require `fortran', not
`gfortran'.
* tests/silentf77.test ($required): Require `fortran77', not
`gfortran'.
* tests/silent-many-generic.test ($required): Require `fortran'
and `fortran77' rather than `gfortran'.
Stefano Lattarini [Fri, 27 May 2011 13:05:40 +0000 (15:05 +0200)]
tests: improve `ccnoco*.test', better cross-compiling support
* tests/ccnoco3.test (Mycomp): Use the `$CC' chosen by `tests/defs'
instead of forcing `gcc' unconditionally. This ensures better
coverage in case of cross-compiling, when GCC can be named e.g.,
`i586-mingw32msvc-gcc', instead of simply `gcc'.
* tests/ccnoco.test: Likewise. Remove redundant checks. Modernize
the created `configure.in'. Run tests both in-tree and in VPATH.
Export `CC' to the overridden value only once.
* tests/ccnoco2.test: Slightly stricter grepping of automake
stderr. Add trailing `:' command.
Stefano Lattarini [Fri, 27 May 2011 08:22:31 +0000 (10:22 +0200)]
Merge branch 'master' into testsuite-work
Stefano Lattarini [Fri, 27 May 2011 08:20:25 +0000 (10:20 +0200)]
testsuite: each test case depends on `defs-static'
* tests/Makefile.am ($(TEST_LOGS)): Depends on `defs-static' too.
Simplify comments.
Stefano Lattarini [Fri, 3 Dec 2010 14:24:14 +0000 (15:24 +0100)]
testsuite: use $SHELL to run tests which are shell scripts
* tests/Makefile.am (TEST_LOG_COMPILER): Define so that the
configure-time $SHELL is used to run the tests.
* tests/defs: Add code to re-execute by default the running
test script with configure-time $SHELL. Updated comments.
* configure.ac: Check that `set -e' is working for $SHELL,
not for /bin/sh.
* tests/defs-static.in: Update comments.
* tests/README (Supported shells): Updated.
(Getting details from failures): Don't tell that tests are
run by /bin/sh by default.
Stefano Lattarini [Thu, 26 May 2011 20:28:45 +0000 (22:28 +0200)]
tests: split 'subst2.test' to improve modularity and coverage
* tests/subst2.test: Test removed, split into ...
* tests/programs-primary-rewritten.test: ... this test ...
* tests/subst-no-trailing-empty-line.test: ... and this one ...
* tests/extra-programs-empty.test: ... and this one.
* tests/Makefile.am (TESTS): Update.
Stefano Lattarini [Thu, 26 May 2011 13:42:02 +0000 (15:42 +0200)]
tests: tweak and improve tests on Automake conditionals
The "avoid the requirement of a working compiler" we refer about
below is obtained by disabling automatic dependency tracking and
defining `CC' (and `EXEEXT', `OBJEXT', `CXX', etc., if needed) to
dummy values, either directly in the `Makefile.am', or by AC_SUBST
in `configure.in'.
The "cosmetic changes" we refer about below consists in, e.g.,
adding a trailing `:' command to a script, using proper m4 quoting
in `configure.in' files, adding commands that offer more debugging
output, explicitly declaring phony targets as such in Makefiles,
making use of idioms more consistent with those employed in other
tests, following the GNU formatting standards more closely, etc.
* tests/cond.test: Cosmetic changes.
* tests/cond3.test: Likewise.
* tests/cond10.test: Likewise.
* tests/cond14.test: Likewise.
* tests/cond15.test: Likewise.
* tests/cond17.test: Likewise.
* tests/cond40.test: Likewise.
* tests/cond13.test: Likewise, and make grepping of the generated
`Makefile.in' slightly stricter.
* tests/cond6.test: Extend by also running ./configure, make and
"make install". Few cosmetic changes.
* tests/cond8.test: Likewise.
* tests/cond2.test: Do few cosmetic changes. Make grepping of
automake stderr stricter.
* tests/cond20.test: Likewise.
* tests/cond23.test: Likewise.
* tests/cond24.test: Likewise.
* tests/cond42.test: Likewise.
* tests/cond46.test: Likewise.
* tests/cond9.test: Move more checks in the `Makefile.am' instead
of grepping make's output.
* tests/cond38.test: Likewise.
* tests/cond11.test: Likewise. Avoid the requirement of a working
C compiler.
* tests/cond16.test: Likewise.
* tests/cond22.test: Likewise.
* tests/cond30.test: Likewise.
* tests/cond31.test: Likewise.
* tests/cond4.test: Likewise, and drop the now-useless GNU make
requirement as well.
* tests/cond18.test: Likewise.
* tests/cond19.test: Likewise.
* tests/cond22.test: Likewise.
* tests/cond25.test: Add trailing `:' command.
* tests/cond26.test: Likewise.
* tests/cond27.test: Likewise.
* tests/cond28.test: Likewise.
* tests/cond29.test: Use `unindent' function for better formatting.
Other minor cosmetic changes. Use a `timeout' program (if it's
available) to determine whether the script takes too long, instead
of just hoping that the user will notice an abnormally long test
execution time.
* tests/cond33.test: Extend a bit. Fix heading comments. Few more
cosmetic changes.
* tests/cond37.test: Ensure verbose printing of captured make
output. Minor cosmetic changes.
* tests/cond34.test: Likewise. Avoid the requirement of a working
C compiler. Make grepping of make output slightly stricter.
* tests/cond35.test: Quote literal dots in grep regexp. Add extra
debugging output. Minor cosmetic changes.
* tests/cond36.test: Likewise.
* tests/cond41.test: Do not simply check that aclocal fails, but
also grep its stderr for the expected error message.
* tests/cond43.test: Likewise (but the program being automake).
Few cosmetic changes.
* tests/condd.test: Avoid the requirement of a working C compiler.
* tests/condman3.test: Extend by using more man pages and more
manpage sections.
* tests/condman.test: Removed, it's completely superseded by
`condman3.test'.
* tests/Makefile.am (TESTS): Update.
Stefano Lattarini [Wed, 25 May 2011 17:05:15 +0000 (19:05 +0200)]
lex tests: avoid spurious failures when LEXLIB isn't found
The AC_PROG_LEX Autoconf macro does not diagnose a failure to find
the "lex library" expected to provide a `yywrap' function (function
which is required to link most lex-generated programs). On the
contrary, when all the link attempts (i.e., with `-ll' and `-lfl')
fail, configure declares that no lex library is needed, and simply
proceeds with the configuration process -- only for the build to
possibly fail later, at make time.
This behaviour might be (partly) intended; the Autoconf manual
reads:
``You are encouraged to use Flex in your sources, since it is
both more pleasant to use than plain Lex and the C source
it produces is portable. In order to ensure portability,
however, you must either provide a function `yywrap' or, if
you don't use it (e.g., your scanner has no `#include'-like
feature), simply include a `%noyywrap' statement in the
scanner's source.''
This AC_PROG_LEX behaviour is causing some spurious failures of the
Automake testsuite in environments which lack a proper library
providing `yywrap' (this happens for example in Linux->MinGW cross
compilations). But at this point is clear that a proper workaround
is to simply provide a fall-back implementation of `yywrap' in our
lexers.
* tests/cond35.test: Provide a dummy `yywrap' function.
* tests/lex3.test: Likewise.
* tests/lexvpath.test: Likewise.
* tests/silent-many-gcc.test: Likewise.
* tests/silent-many-generic.test: Likewise.
* tests/silent-lex-gcc.test: Likewise, and a dummy `main' too.
* tests/silent-lex-generic.test: Likewise.
* tests/lex-lib.test: New test.
* tests/lex-libobj.test: New test.
* tests/lex-nowrap.test: New test.
* tests/Makefile.am (TESTS): Update.
* THANKS: Update.
Thanks to Russ Allbery for the suggestion.
Stefano Lattarini [Wed, 25 May 2011 14:45:30 +0000 (16:45 +0200)]
testsuite: require C++ compiler explicitly in tests needing it
The list of the test scripts which needed a C++ compiler but
failed to require it explicitly has been found by running:
$ tests=`grep -lE 'CXX|\.c(c|\+\+|xx|pp)' *.test | tr '\n' ' '`
$ am_explicit_skips=yes CXX=false make check TESTS="$tests"
and looking for tests that reported FAIL instead of SKIP.
After the present change, all those previously failing tests
either pass or get correctly skipped.
* tests/subobj9.test: Require `c++', instead of explicitly
skipping on configure failure. Add excerpts and/or details from
the original bug report that prompted this tests to be written.
* tests/silentcxx-gcc.test: Require g++.
* tests/silentcxx.test: Require c++.
* tests/suffix3.test: Likewise.
Stefano Lattarini [Wed, 25 May 2011 14:15:27 +0000 (16:15 +0200)]
testsuite: don't require g++ where any C++ compiler is enough
* tests/specflg10.test ($required): Use `c++', not g++.
* tests/silent-many-generic.test: Likewise.
Stefano Lattarini [Mon, 23 May 2011 19:53:44 +0000 (21:53 +0200)]
testsuite: require C compiler explicitly in tests needing it
The list of the test scripts which needed a C compiler but
failed to require it explicitly has been found by running:
$ am_explicit_skips=yes CC=false make check
and looking for tests that reported FAIL instead of SKIP.
After the present change, all those previously failing tests
either pass or get correctly skipped.
* tests/aclocal4.test ($required): Add `cc'.
* tests/ansi.test: Likewise.
* tests/ansi10.test: Likewise.
* tests/ansi3.test: Likewise.
* tests/ansi3b.test: Likewise.
* tests/ansi5.test: Likewise.
* tests/ansi6.test: Likewise.
* tests/ansi7.test: Likewise.
* tests/ansi9.test: Likewise.
* tests/backcompat6.test: Likewise.
* tests/canon7.test: Likewise.
* tests/check5.test: Likewise.
* tests/check7.test: Likewise.
* tests/check8.test: Likewise.
* tests/cond4.test: Likewise.
* tests/cond11.test: Likewise.
* tests/cond16.test: Likewise.
* tests/cond18.test: Likewise.
* tests/cond19.test: Likewise.
* tests/cond22.test: Likewise.
* tests/cond30.test: Likewise.
* tests/cond31.test: Likewise.
* tests/cond32.test: Likewise.
* tests/cond34.test: Likewise.
* tests/cond35.test: Likewise.
* tests/cond39.test: Likewise.
* tests/condd.test: Likewise.
* tests/cygnus-dependency-tracking.test: Likewise.
* tests/depcomp2.test: Likewise.
* tests/depcomp6.test: Likewise.
* tests/depcomp7.test: Likewise.
* tests/depcomp8a.test: Likewise.
* tests/depcomp8b.test: Likewise.
* tests/depdist.test: Likewise.
* tests/depend2.test: Likewise.
* tests/depend5.test: Likewise.
* tests/distcleancheck.test: Likewise.
* tests/distname.test: Likewise.
* tests/exeext.test: Likewise.
* tests/exeext4.test: Likewise.
* tests/extradep.test: Likewise.
* tests/extradep2.test: Likewise.
* tests/gnits2.test: Likewise.
* tests/gnits3.test: Likewise.
* tests/instdir-ltlib.test: Likewise.
* tests/instdir-prog.test: Likewise.
* tests/instfail.test: Likewise.
* tests/instfail-libtool.test: Likewise.
* tests/lex3.test: Likewise.
* tests/lex5.test: Likewise.
* tests/lexvpath.test: Likewise.
* tests/lex-subobj-nodep.test: Likewise.
* tests/lflags.test: Likewise.
* tests/libobj-basic.test: Likewise.
* tests/libobj2.test: Likewise.
* tests/libobj7.test: Likewise.
* tests/libobj10.test: Likewise.
* tests/libobj16a.test: Likewise.
* tests/libobj16b.test: Likewise.
* tests/libobj17.test: Likewise.
* tests/libobj19.test: Likewise.
* tests/libtool2.test: Likewise.
* tests/libtool3.test: Likewise.
* tests/libtool7.test: Likewise.
* tests/libtool9.test: Likewise.
* tests/libtoo10.test: Likewise.
* tests/libtoo11.test: Likewise.
* tests/ltcond.test: Likewise.
* tests/ltcond2.test: Likewise.
* tests/ltconv.test: Likewise.
* tests/ltinit.test: Likewise.
* tests/ltlibsrc.test: Likewise.
* tests/ltorder.test: Likewise.
* tests/nobase.test: Likewise.
* tests/nobase-libtool.test: Likewise.
* tests/mmodely.test: Likewise.
* tests/parallel-tests5.test: Likewise.
* tests/parallel-tests-suffix-prog.test: Likewise.
* tests/parallel-tests-ext-driver-prog.test: Likewise.
* tests/posixsubst-ldadd.test: Likewise.
* tests/posixsubst-libraries.test: Likewise.
* tests/posixsubst-ltlibraries.test: Likewise.
* tests/posixsubst-programs.test: Likewise.
* tests/posixsubst-sources.test: Likewise.
* tests/pr87.test: Likewise.
* tests/pr204.test: Likewise.
* tests/pr224.test: Likewise.
* tests/pr300-lib.test: Likewise.
* tests/pr300-ltlib.test: Likewise.
* tests/pr300-prog.test: Likewise.
* tests/pr401.test: Likewise.
* tests/pr401b.test: Likewise.
* tests/pr401c.test: Likewise.
* tests/remake-gnulib-add-acsubst.test: Likewise.
* tests/remake-gnulib-add-header.test: Likewise.
* tests/regex.test: Likewise.
* tests/repeated-options.test: Likewise.
* tests/silent.test: Likewise.
* tests/silent3.test: Likewise.
* tests/silent9.test: Likewise.
* tests/silent-lex-generic.test: Likewise.
* tests/silent-many-generic.test: Likewise.
* tests/silent-yacc-generic.test: Likewise.
* tests/specflg7.test: Likewise.
* tests/specflg9.test: Likewise.
* tests/specflg10.test: Likewise.
* tests/stdinc.test: Likewise.
* tests/strip.test: Likewise.
* tests/strip2.test: Likewise.
* tests/strip3.test: Likewise.
* tests/subdirbuiltsources.test: Likewise.
* tests/subobj3.test: Likewise.
* tests/subobj6.test: Likewise.
* tests/subobj11a.test: Likewise.
* tests/subpkg.test: Likewise.
* tests/subst2.test: Likewise.
* tests/subst3.test: Likewise.
* tests/substref.test: Likewise.
* tests/substre2.test: Likewise.
* tests/suffix5.test: Likewise.
* tests/suffix8.test: Likewise.
* tests/suffix10.test: Likewise.
* tests/suffix11.test: Likewise.
* tests/suffix12.test: Likewise.
* tests/suffix13.test: Likewise.
* tests/target-cflags.test: Likewise.
* tests/transform.test: Likewise.
* tests/transform2.test: Likewise.
* tests/yacc-basic.test: Likewise.
* tests/yacc-d-basic.test: Likewise.
* tests/yacc-clean.test: Likewise.
* tests/yacc-dist-nobuild.test: Likewise.
* tests/yacc-nodist.test: Likewise.
* tests/yaccvpath.test: Likewise.
* tests/yacc-d-vpath.test: Likewise.
* tests/yacc4.test: Likewise.
* tests/yacc7.test: Likewise.
* tests/yacc8.test: Likewise.
* tests/yaccdry.test: Likewise.
* tests/yflags-cmdline-override.test: Likewise.
* tests/yflags-force-override.test: Likewise.
* tests/python-virtualenv.test: Likewise. Also, improve skip
messages.
* tests/subobj5.test ($required): Add `cc'.
(Makefile.am): Set `AUTOMAKE_OPTIONS' to `subdir-objects', and
add new checking rules `test-build' and `test-distdir'.
Extend the test by building and examining the distdir, the
program, and the object files.
* tests/postproc.test ($required): Add `cc'.
Avoid the explicit `|| Exit $?' after call to configure, which
is now either redundant (e.g., when `am_explicit_skips' is
unset), or counter-productive (e.g., when `am_explicit_skips'
is set to "yes").
* tests/pr243.test: Likewise.
* tests/pr266.test: Likewise.
* tests/pr220.test: Simplify so that it doesn't require a
C compiler anymore.
* tests/subdir5.test: Likewise.
* tests/subdir8.test: Likewise.
* tests/lflags.test: Likewise.
* tests/yflags.test: Likewise.
* tests/yflags-force-conditional.test: Likewise.
* tests/lflags2.test: Simplify so that it doesn't require a
C++ compiler anymore.
* tests/yflags2.test: Likewise.
* tests/autohdrdry.test (configure.in): Remove unneeded call
to `AC_PROG_CC'.
* tests/pr287.test: Likewise.
* tests/check6.test: Likewise.
* tests/cond21.test: Likewise, plus some cosmetic adjustments.
* tests/upc.test: Skip if configure fails with status `77'.
* tests/upc3.test: Likewise.
* tests/vala4.test: Likewise.
* tests/nostdinc.test: Likewise. Also, make grepping checks
on `Makefile.in'. Update heading comments.
* tests/compile5.test: When we must skip, skip explicitly,
and with a meaningful message.
* tests/instspc-tests.sh: When running in "test-build" or
"test-install" mode (as determined by the value of variable
`$instspc_action'), require `cc'. Adjust comments.
* tests/depmod-tests.sh: Likewise.
Stefano Lattarini [Mon, 23 May 2011 13:51:31 +0000 (15:51 +0200)]
testsuite: allow user to ask for stricter skip semantics
Given how the Automake testsuite is currently structured, if a
command in a test script fails with status `77', the global test
outcome is considered a SKIP, because the value of `77' for `$?'
is passed to the exit trap. Indeed, this happens in practice, as
an autoconf-generated configure script can exit with status `77'
if it fails to find, e.g., a required compiler.
While this behaviour is quite useful for avoiding spurious test
failures in the wild, it can also sometimes prevent the Automake
developers to easily see and declare the requirements of their
tests.
This change introduces a new variable `am_explicit_skips', meant to
be user-overridable, and which, when set to a "true" value (i.e.,
`yes' or `1'), require a direct call to `Exit 77' in order to make
the test outcome be considered a SKIP.
* tests/defs.in ($am__test_skipped): New variable, initialized
to `no'.
(Exit): Set `$am__test_skipped' to `yes' if passed an exit status
of 77.
(trap '...' 0): When `$am_explicit_skips' is set to a "true" value,
reset an exit status of `77' to `78' if $am__test_skipped is not
set to `yes'.
* tests/self-check-exit.test: Adjust: unset `am_explicit_skips'.
* tests/self-check-explicit-skips.test: New test.
* tests/Makefile.am (TESTS): Update.
Stefano Lattarini [Tue, 24 May 2011 08:42:27 +0000 (10:42 +0200)]
testsuite: user can force skipping of compiler-requiring tests
* tests/defs (cc, c++, fortran, fortran77): Skip the test if
the relevant compiler is disabled by having the corresponding
variable (CC, CXX, FC and F77, respectively) set to "false".
(yacc): For consistency, skip the test when the YACC variable
is set to "false", not when it's set to "no". Since we are at
it, fix the skip message to be shorter and more consistent.
Stefano Lattarini [Mon, 23 May 2011 12:26:58 +0000 (14:26 +0200)]
tests: tweak and improve tests on "Simple Tests" driver
* tests/check.test: Make grepping of generated Makefile.in
slightly stricter. Add trailing `:' command.
* tests/check2.test: Ensure verbose printing of captured make
output. Make grepping of generated Makefile.in and of make
output stricter. Add trailing `:' command.
* tests/check3.test: Likewise. Also, fix minor error in a
comment.
* tests/check4.test: Stricter grepping of make output. Use
`using_gmake' subroutine instead of copy&paste from defs.in
to determine whether $MAKE is GNU make. Add a trailing `:'
command.
* tests/check6.test: Add trailing `:' command.
* tests/check10.test: Prefer redirection `>' over `>>' when
creating new files. Ensure verbose printing of captured make
output.
* tests/check11.test: Likewise.
* tests/check7.test: Likewise. Make grepping of make output
stricter. Prefer "make distcheck" over a simple "make check"
in a VPATH build. Add trailing `:' command.
(a.c): Format following GNU coding standards.
(Makefile.am): Explicitly declare target `print-xfail-tests'
as PHONY. Distribute $(check_SCRIPTS), for distcheck.
* tests/parallel-tests.test: Prefer redirection `>' over `>>'
when creating new files.
* tests/parallel-tests2.test: Likewise.
* tests/parallel-tests9.test: Likewise.
* tests/parallel-tests8.test: Likewise. Make grepping of
automake stderr stricter.
* tests/check8.test: Likewise. Make formatting of created
*.c files more consistent with GNU Coding Standards.
* tests/check-subst-prog.test (configure.in): Break overly
long lines.
(pass-prog.c, xfail-prog.c): Format them more consistently
with GNU Coding Standards.
* tests/parallel-tests3.test: Use a shorter and clearer skip
message. Use the `unindent' subroutine to improve readability
of indented loop bodies. Simplify the hack used to please
maintainer-check.
* tests/parallel-tests5.test: Make formatting of created *.c
files more consistent with GNU Coding Standards. Use the
`unindent' subroutine to improve readability of indented loop
bodies.
* tests/parallel-tests10.test: Add blank line, for clarity.
Prefer trailing `:' over trailing `Exit 0', for consistency
with other tests.
Ralf Wildenhues [Sun, 22 May 2011 17:02:27 +0000 (19:02 +0200)]
tests/README: fix example about `make -e' usage
* tests/README (Section "Writing test cases" subsection "Do"): When
some variable is never initialized in the Makefile, `-e' is not
necessary in order to override it. DESTDIR is such a variable: we
ensure that we do not ever initialize it. And as such, it is quite
portable to use:
$ make DESTDIR=/foo/bar install
and in fact, quite widely used.
So our example about when `make -e' is required, which references
the `DESTDIR' variable, is poorly chosen, if not downright wrong.
Rewrite it to use `prefix' as the overridden variable instead.
Stefano Lattarini [Sat, 21 May 2011 09:03:00 +0000 (11:03 +0200)]
tests/README: small consistency improvement
* tests/README (Section "User interface" subsection "Running the
tests"): In examples, be more consistent w.r.t. the differences
between GNU and non-GNU make.
Stefano Lattarini [Sat, 21 May 2011 08:59:53 +0000 (10:59 +0200)]
tests/README: suggest how to run tests in cross-compile mode
* tests/README (Section "User interface" subsection "Running the
tests"): Briefly explain how to override 'host_alias' at runtime
to force the use of cross-compilers by the testsuite. Give an
example.
Stefano Lattarini [Sat, 21 May 2011 07:12:06 +0000 (09:12 +0200)]
test defs: better detection of cross-compile mode
* configure.ac (AC_CANONICAL_HOST): New, probably not strictly
necessary, but useful to complements AC_CANONICAL_BUILD.
* tests/defs-static.in ($host_alias, $build_alias): New variables,
exported, user-overridable, defaults AC_SUBST'd by configure.
* tests/defs (cross_compiling): Consider cross-compilation mode
active only if `$host_alias' is not empty *and* it differs from
`$build_alias'. This is more faithful to the way configure
determines whether it is cross-compiling, but still allows the
user to easily force cross-compilation by overriding `host_alias'
and `build_alias' at runtime.
From a report by Ralf Wildenhues.
Stefano Lattarini [Fri, 20 May 2011 22:08:02 +0000 (00:08 +0200)]
Merge branch 'master' into testsuite-work
* master:
testsuite: avoid re-running few tests with 'parallel-tests' option
testsuite: allow tests to avoid the use of 'parallel-tests' option
tests/README: update obsoleted advice
Stefano Lattarini [Fri, 20 May 2011 21:33:09 +0000 (23:33 +0200)]
Merge branch 'maint'
* maint:
testsuite: avoid re-running few tests with 'parallel-tests' option
testsuite: allow tests to avoid the use of 'parallel-tests' option
tests/README: update obsoleted advice
Stefano Lattarini [Fri, 20 May 2011 21:18:39 +0000 (23:18 +0200)]
testsuite: avoid re-running few tests with 'parallel-tests' option
Some tests in our testsuite use the 'simple-tests' driver only
marginally, or simply as a mean to conveniently check unrelated
invariants. It makes little sense to force these tests to also
run with the 'parallel-tests' Automake option active, as doing so
offers no real gain in coverage, while often causing a measurable
overhead in execution time (for an already too-slow testsuite).
* tests/pr401.test (parallel_tests): Define to "no", to prevent
the generation of a sibling test script using the 'parallel-tests'
driver.
* tests/pr401b.test: Likewise.
* tests/pr401c.test: Likewise.
Stefano Lattarini [Fri, 20 May 2011 20:31:40 +0000 (22:31 +0200)]
testsuite: allow tests to avoid the use of 'parallel-tests' option
* tests/gen-parallel-tests: Do not generate "siblings" for tests
that explicitly define the `parallel_tests', whether to "yes" or
to any other value. Extend heading comments to give a rationale
for this behaviour.
* tests/README: Update.
Stefano Lattarini [Fri, 20 May 2011 20:15:32 +0000 (22:15 +0200)]
tests/README: update obsoleted advice
* tests/README (Section "Writing test cases" subsection "Do"):
Do not suggest to use the `*-p.test' pattern for the names of
hand-written tests which use the `parallel-tests' Automake option.
Not only is this not respected by the existing tests, but it is
more likely to cause conflicts with auto-generated tests.
So, suggest to *avoid* using the `*-p.test' pattern in names
of hand-written tests instead.
(Section "Writing test cases" subsection "Do not"): When
suggesting not to override Makefile variables using command
line arguments, do not use the badly outdated variables `U'
and 'ANSI2KNR' in the example; instead, use the more common
and typical `DESTDIR'.
Stefano Lattarini [Thu, 19 May 2011 20:50:35 +0000 (22:50 +0200)]
fix typos in latest ChangeLog entry
Stefano Lattarini [Sun, 15 May 2011 14:33:01 +0000 (16:33 +0200)]
tests: avoid spurious failures in cross-compile mode
* tests/depcomp2.test: Ensure verbose printing of captured stderr
from configure.
* tests/ansi3.test ($required): Add 'native', as the ansi2knr
functionality is not meant to work with a cross-compiler.
* tests/ansi3b.test: Likewise.
* tests/ansi5.test: Likewise.
* tests/ansi6.test: Likewise.
* tests/ansi7.test: Likewise.
* tests/ansi8.test: Likewise.
* tests/ansi10.test: Likewise.
* tests/subobj3.test: Likewise.
* tests/check8.test ($required): Add 'native', as this test is
not meant to work with a cross-compiler.
* tests/gnits2.test: Likewise.
* tests/ltconv.test: Likewise.
* tests/ltcond2.test: Likewise.
* tests/parallel-tests5.test: Likewise.
* tests/specflg7.test: Likewise.
* tests/specflg8.test: Likewise.
* tests/check5.test: Likewise. Also, ensure verbose printing of
captured make output, and thrown in few cosmetic and consistency
improvements.
* tests/ltinit.test: Be laxer in grepping configure output, to
avoid spurious failures on systems which lack POSIX dynamic
linking (e.g., MinGW), or when cross-compiling for such systems.
* tests/ansi9.test: Skip parts of the test that would require a
native compiler.
* tests/backcompat6.test: Likewise.
* tests/depcomp8a.test: Likewise.
* tests/pr401.test: Likewise.
* tests/pr401b.test: Likewise.
* tests/pr401c.test: Likewise.
* tests/target-cflags.test: Likewise.
* tests/lex3.test: Likewise.
* tests/yacc-basic.test: Likewise.
* tests/autohdr4.test: Likewise. Also, extend the test to ensure
more coverage in cross-compile mode.
* tests/canon7.test: Likewise.
* tests/check9.test: Removed, split into two new tests ...
* tests/check-subst-prog.test: ... this one, which requires a
native C compiler ...
* tests/check-subst.test: ... and this one, which doesn't require
any compiler.
* tests/parallel-tests4.test: Removed, split into two new tests ...
* tests/parallel-tests-suffix-prog.test: ... this one, which
requires a native C compiler ...
* tests/parallel-tests-suffix.test: ... and this one, which
doesn't require any compiler.
* tests/parallel-tests7.test: Removed, split into two new tests ...
* tests/parallel-tests-ext-driver-prog.test: ... this one, which
requires a native C compiler ...
* tests/parallel-tests-ext-driver.test: ... and this one, which
doesn't require any compiler.
* tests/Makefile.am (TESTS): Update.
Stefano Lattarini [Thu, 19 May 2011 20:07:48 +0000 (22:07 +0200)]
Merge branch 'master' into testsuite-work
* master:
test defs: rename requirement 'non-cross' -> 'native'
Stefano Lattarini [Thu, 19 May 2011 19:53:02 +0000 (21:53 +0200)]
Merge branch 'maint'
* maint:
test defs: rename requirement 'non-cross' -> 'native'
Stefano Lattarini [Thu, 19 May 2011 19:48:27 +0000 (21:48 +0200)]
test defs: rename requirement 'non-cross' -> 'native'
* tests/defs.in (non-cross): Rename requirement ...
(native): ... to this, which is clearer and fits the
existing lingo better.
Suggestion by Ralf Wildenhues.
Stefano Lattarini [Sun, 15 May 2011 13:38:14 +0000 (15:38 +0200)]
Merge branch 'master' into testsuite-work
* master:
testsuite: be more cross-compile friendly
tests: fix portability issues in 'repeated-options.test'
Stefano Lattarini [Sun, 15 May 2011 13:35:27 +0000 (15:35 +0200)]
Merge branch 'maint'
* maint:
testsuite: be more cross-compile friendly
Stefano Lattarini [Sun, 15 May 2011 12:30:43 +0000 (14:30 +0200)]
testsuite: be more cross-compile friendly
* tests/defs.in (cross_compiling): New subroutine.
(am__tool_prefix): New internal variable.
(gcc, g++, gcj): Force the use of the correct "tool prefix"
when cross compiling.
(gfortran, g77, non-cross): New requirements.
Stefano Lattarini [Sun, 15 May 2011 12:41:57 +0000 (14:41 +0200)]
Merge branch 'test-repeated-options-portability-fix'
Stefano Lattarini [Sun, 15 May 2011 12:41:35 +0000 (14:41 +0200)]
tests: fix portability issues in 'repeated-options.test'
* tests/repeated-options.test: Do not assume that object files
have `.o' suffix and executables have no default suffix; instead,
use `$(OBJEXT)' and `$(EXEEXT)'.
Stefano Lattarini [Sat, 7 May 2011 13:40:54 +0000 (15:40 +0200)]
Merge branch 'master' into testsuite-work
* master:
tests: fix spurious failure of extradep.test on FreeBSD
Stefano Lattarini [Sat, 7 May 2011 13:39:37 +0000 (15:39 +0200)]
Merge branch 'maint'
* maint:
tests: fix spurious failure of txinfo21.test on FreeBSD
Stefano Lattarini [Sat, 7 May 2011 13:38:56 +0000 (15:38 +0200)]
Merge branch 'maint' into testsuite-work
* maint:
tests: fix spurious failure of txinfo21.test on FreeBSD
Stefano Lattarini [Sat, 7 May 2011 13:35:35 +0000 (15:35 +0200)]
Merge branch 'test-silentcxx-gcc-fix' into testsuite-work
Stefano Lattarini [Sat, 7 May 2011 12:18:47 +0000 (14:18 +0200)]
tests: fix syntax error in 'silentcxx-gcc.test'
* tests/silentcxx-gcc.test (for config_args in ...): Remove extra
trailing backslash from the list of looped-on items.
Stefano Lattarini [Sat, 7 May 2011 13:23:57 +0000 (15:23 +0200)]
Merge branch 'test-extradep-fix'
Stefano Lattarini [Sat, 7 May 2011 13:23:28 +0000 (15:23 +0200)]
typofix in ChangeLog
Stefano Lattarini [Sat, 7 May 2011 13:10:14 +0000 (15:10 +0200)]
Merge branch 'test-extradep-fix'
Stefano Lattarini [Sat, 7 May 2011 10:58:51 +0000 (12:58 +0200)]
tests: fix spurious failure of extradep.test on FreeBSD
* tests/extradep.test: When using `ls -t' to determine whether a
file has been updated, make sure to use as reference a file whose
timestamp is expected to be *strictly* older that that of the file
being checked. This is required because at least FreeBSD `ls' do
not sort files with the same timestamp in alphabetical order when
using the `-t' option.
* tests/extradep2.test: Likewise.
Stefano Lattarini [Sat, 7 May 2011 12:34:39 +0000 (14:34 +0200)]
tests: fix spurious failure of txinfo21.test on FreeBSD
* tests/txinfo21.test: Use the `is_newest' subroutine instead of
the `ls -t' hack to to determine whether a file has been updated.
This is required because at least FreeBSD `ls' do not sort files
with the same timestamp in alphabetical order when using the `-t'
option.
Stefano Lattarini [Sat, 7 May 2011 09:50:02 +0000 (11:50 +0200)]
Merge branch 'fix-depmod-tests-generation' into testsuite-work
Stefano Lattarini [Sat, 7 May 2011 09:49:40 +0000 (11:49 +0200)]
depmod tests: fix bug in depmodes list extraction
* tests/depmod-tests.sh (get_depmodes): Use `echo $all_depmodes'
instead of `... | tr "$nl" "$sp"' to strip extra whitespaces and
newlines from `$all_depmodes'. The latter idiom with `tr' is
wrong since it gets run before `./defs' is sourced, and thus `$nl'
and `$sp' are undefined.
Bug revealad by FreeBSD tr(1) implementation, which doesn't accept
empty strings as arguments.
Stefano Lattarini [Wed, 4 May 2011 21:36:25 +0000 (23:36 +0200)]
tests: add excerpts from few original bug reports, for clarity.
* tests/comment6.test: Add excerpts and/or details from the
original bug report that prompted this tests to be added and/or
extended.
* tests/dejagnu4.test: Likewise.
* tests/installdir.test: Likewise.
Stefano Lattarini [Wed, 4 May 2011 21:09:54 +0000 (23:09 +0200)]
Merge branch 'master' into testsuite-work
Stefano Lattarini [Wed, 4 May 2011 21:06:16 +0000 (23:06 +0200)]
ChangeLog: some typofixes
Stefano Lattarini [Wed, 4 May 2011 20:50:59 +0000 (22:50 +0200)]
Merge branch 'maint'
Stefano Lattarini [Wed, 4 May 2011 20:42:19 +0000 (22:42 +0200)]
ChangeLog: various typofixes (and related reformatting)
Stefano Lattarini [Wed, 4 May 2011 20:21:53 +0000 (22:21 +0200)]
Merge branch 'maint'
With some edits to `tests/defs' and `ChangeLog', to reflect the
fact that in master we already had a `yacc' requirement, and no
more had an explicitly-recognized `bison' requirement.
Stefano Lattarini [Wed, 4 May 2011 16:42:11 +0000 (18:42 +0200)]
tests defs: allow requirements for compilers (mostly dummy)
Most of the new requirements that are now accepted in `$required'
as consequence of this patch are still dummy. They are planned
to be implemented only in master (or in some derived branch), but
having them here (even just as no-op) will allow for an easier
integration/backporting of potential new testcases.
* tests/defs.in (cc, c++, fortran, fortran77): New requirements,
still dummy.
(flex): New requirement, picking LEX for configure.
(lex): New requirement, alias for `flex'. A more appropriate
implementation, looking for a generic `lex' program, will follow
in the future.
(yacc): New requirement, alias for `bison'. A more appropriate
implementation, looking for a generic `yacc' program, will follow
in the future.
Stefano Lattarini [Wed, 4 May 2011 09:58:21 +0000 (11:58 +0200)]
tests: 'silent-many-gcc.test' improved and made more robust
* tests/silent-many-gcc.test: Also force "fast" gcc depmode
for C++ compilations. Add sanity checks verifying that the
cache variables we force are really used by configure. Fix
typo in comments.
Stefano Lattarini [Wed, 4 May 2011 09:48:19 +0000 (11:48 +0200)]
tests: drop useless requirements of 'g++'
* tests/lflags2.test ($required): Remove 'g++', as any working
C++ compiler should be enough, and ./configure will cause the
test to skip if no working C++ compiler is found.
* tests/yflags2.test: Likewise.
* tests/subobj9.test: Likewise.
* tests/silentcxx.test: Likewise. Also, do not force depmodes
that could cause non-GNU C++ compilers to fail.
* tests/silentcxx-gcc.test: New test, like `silentcxx.test',
but forcing "fast" gcc depmode (and thus requiring the GNU C++
compiler).
* tests/specflg10.test: Add proper "fixme" comment telling that
we should make this test work with a generic C++ compiler.
* tests/Makefile.am (TESTS): Update.
Stefano Lattarini [Tue, 3 May 2011 21:40:00 +0000 (23:40 +0200)]
tests defs: one more environment sanitization (corner case)
* tests/defs (am__using_gmake): Initialize. The `using_gmake'
subroutine was using this variable for caching, but wasn't
initializing it, which could cause problems in the (admittedly
very unlikely) case in which it was pre-existent in the
environment.
Stefano Lattarini [Tue, 3 May 2011 17:23:31 +0000 (19:23 +0200)]
Merge branch 'master' into testsuite-work
Stefano Lattarini [Tue, 3 May 2011 17:21:28 +0000 (19:21 +0200)]
Merge branch 'tests-man-fixes'
Stefano Lattarini [Tue, 3 May 2011 17:19:13 +0000 (19:19 +0200)]
tests: add forgotten test scripts to $(TESTS)
* tests/Makefile.am (TESTS): Added test scripts man[678].test,
which have been present on the filesystem and in the repository,
but which had erroneously been left out from $(TESTS).
Stefano Lattarini [Tue, 3 May 2011 17:08:30 +0000 (19:08 +0200)]
Merge branch 'check-tests-list' into testsuite-work
Stefano Lattarini [Sat, 30 Jan 2010 00:23:44 +0000 (01:23 +0100)]
maintcheck: consistency of list of test scripts
* CheckListOfTests.am: New file.
(maintainer-check-list-of-tests): New target,
check for consistency between list of tests defined in the
including Makefile and list of tests on the filesystem.
(clean-maintcheck-testslist-tmp): New rule, to clean up the
temporary files that might be left around by the rules associated
with the previous target.
(clean-local): Depend on it.
* lib/Automake/tests/Makefile.am: Include `CheckListOfTests.am'.
(maintainer-check): Added dependency from the new target
`maintainer-check-list-of-tests'.
* tests/Makefile.am: Likewise.
* Makefile.am (maintainer-check-list-of-test): New target,
calling recursively into `tests/' and `lib/Automake/tests/',
using ...
(TEST_SUBDIRS): ... this new variable.
Stefano Lattarini [Tue, 3 May 2011 16:24:29 +0000 (18:24 +0200)]
tests: ensure verbose printing of captured make output
* tests/libtool7.test: Ensure verbose printing of captured
make output.
* tests/libtool9.test: Likewise.
Stefano Lattarini [Tue, 3 May 2011 15:51:42 +0000 (17:51 +0200)]
tests: drop useless requirements of gcc (plus testsuite tweakings)
* tests/libobj14.test ($required): Remove 'gcc', as we never run
the C compiler.
* tests/targetclash.test: Likewise.
* tests/ansi6.test ($required): Remove 'gcc', as any working
C compiler should be enough.
* tests/ansi7.test: Likewise.
* tests/ansi9.test: Likewise.
* tests/ansi10.test: Likewise.
* tests/lex5.test: Likewise.
* tests/lexvpath.test: Likewise.
* tests/mmodely.test: Likewise.
* tests/pr204.test: Likewise.
* tests/pr300-lib.test: Likewise.
* tests/pr300-prog.test: Likewise.
* tests/primary3.test: Likewise.
* tests/specflg7.test: Likewise.
* tests/specflg8.test: Likewise.
* tests/subdir5.test: Likewise.
* tests/subdir8.test: Likewise.
* tests/subobj6.test: Likewise.
* tests/subst3.test: Likewise.
* tests/substre2.test: Likewise.
* tests/yacc6.test: Likewise.
* tests/yacc8.test: Likewise.
* tests/depcomp2.test: Likewise. Also, avoid clobbering user-set
CFLAGS.
* tests/lex3.test: Likewise.
* tests/ansi3.test: Likewise. Also, avoid 'CC=gcc' in configure.
* tests/ansi3b.test: Likewise.
* tests/ansi5.test: Likewise.
* tests/autohdr4.test ($required): Remove 'gcc', as any working
C compiler should be enough.
Also, do not reject slow dependency extractors (which we might
be forced to use now that $CC is not necessarily gcc anymore).
* tests/cond16.test ($required): Remove 'gcc', as any working
C compiler should be enough.
Since we are at it, throw in few minor tweakings (mostly cosmetic,
stylistic, or consistency-related).
* tests/cond18.test: Likewise.
* tests/cond35.test: Likewise.
* tests/gnits2.test: Likewise.
* tests/libtool3.test: Likewise.
* tests/libtool7.test: Likewise.
* tests/libtool9.test: Likewise.
* tests/ltcond.test: Likewise.
* tests/ltcond2.test: Likewise.
* tests/ltconv.test: Likewise.
* tests/ltlibsrc.test: Likewise.
* tests/nobase.test: Likewise.
* tests/nobase-libtool.test: Likewise.
* tests/pr220.test: Likewise.
* tests/pr224.test: Likewise.
* tests/pr300-ltlib.test: Likewise.
* tests/pr401.test: Likewise.
* tests/pr401b.test: Likewise.
* tests/pr401c.test: Likewise.
* tests/subpkg.test: Likewise.
* tests/target-cflags.test: Likewise.
* tests/transform.test: Likewise.
* tests/yacc4.test: Likewise.
* tests/cond19.test: Likewise. Also, avoid clobbering user-set
CFLAGS.
* tests/cond4.test: Likewise.
* tests/depend2.test: Likewise.
* tests/pr87.test: Likewise.
* tests/subobj3.test: Likewise.
* tests/substref.test: Likewise.
Stefano Lattarini [Tue, 3 May 2011 10:00:10 +0000 (12:00 +0200)]
test defs: increase coverage w.r.t. GNU make
* tests/defs (GNUmake): Instead of just skipping the tests
requiring GNU make if $MAKE is not GNU make, try to look for
it and, if found, redefine $MAKE accordingly. This will help
to transparently increase coverage on non-GNU systems which
have GNU make available in PATH.
Stefano Lattarini [Tue, 3 May 2011 09:29:44 +0000 (11:29 +0200)]
tests: improve few inter-tests references
* tests/acloca22.test: Improve and extend the heading comments.
Add reference to related tests 'remake-deleted-m4-file.test' and
'remake-renamed-m4-macro-and-file.test'. Since we are at it,
add trailing `:' command, and few blank lines for readability.
* tests/remake-renamed-m4-macro-and-file.test: In the heading
comments, add reference to the related test 'acloca22.test'.
Stefano Lattarini [Tue, 3 May 2011 09:06:30 +0000 (11:06 +0200)]
Merge branch 'master' into testsuite-work
Stefano Lattarini [Mon, 2 May 2011 17:04:01 +0000 (19:04 +0200)]
tests: few more checks on automatic remake support
* tests/remake-all-1.test: New test, check that the "all" target
triggers rebuilt of outdated Makefiles.
* tests/remake-all-2.test: Likewise, but for when the makefiles
are not named `Makefile'.
* tests/Makefile.am (TESTS): Update.
Stefano Lattarini [Mon, 2 May 2011 16:05:34 +0000 (18:05 +0200)]
tests: check some remake features with non-GNU make too
* tests/acloca14.test ($required): Drop "GNUmake". This test
works as-is with non-GNU make implementations.
* tests/remake1a.test: Likewise.
* tests/aclocal4.test ($required): Drop "GNUmake".
Modify the test to have it work also with non-GNU make.
* tests/remake5.test: Likewise.
* tests/remake8a.test: Likewise.
* tests/remake8b.test: Likewise.
* tests/remake9a.test: Likewise.
* tests/remake9b.test: Likewise.
* tests/remake9c.test: Likewise.
* tests/remake9d.test: Likewise.
* tests/remake10a.test: Likewise.
* tests/remake10b.test: Likewise.
* tests/remake10c.test: Likewise.
* tests/remake12.test: ($required): Drop "GNUmake".
Adapt the test to make it work also with non-GNU make (if
it supports an "include" directive).
Stefano Lattarini [Mon, 2 May 2011 22:27:07 +0000 (00:27 +0200)]
tests: fix spurious failure in 'color2.test'
* tests/color2.test: If $MAKE contains command-line arguments (as
in e.g., "make -j2"), expect's directive "spawn $env(MAKE)" fails
spuriously, because it tries to run "$MAKE" as a single command.
Fix this with proper uses of the TCL `eval' builtin.
Stefano Lattarini [Sun, 17 Apr 2011 12:28:17 +0000 (14:28 +0200)]
tests: extend tests on dependency tracking with VPATH
* tests/depmod-tests.sh: New script, fulfilling a threefold role:
1. it is called to generate a Makefile.am snippet, containing the
definition of proper lists of tests;
2. it is called to set up a directory containing some common data
files and autotools-generated files used by the aforementioned
tests (this is done for speed reasons only); and
3. it is called to properly run those tests, one at a time.
* tests/depcomp9.test: Delete, it's obsolete now.
* tests/depcomp10.test: Likewise.
* tests/Makefile.am ($(srcdir)/depmod-tests.am): Include this
snippet, which defines ...
(depmode_tests): ... this new macro, containing the list of the
newly generated `*.depmod' tests.
(TESTS_EXTENSIONS): Add `.depmod'.
(DEPMOD_LOG_COMPILER): Define. It calls `depmod-tests.sh', so that
the "depmode tests" will be executed by passing that driver script
a proper parameter.
($(depmod_tests)): New dependency declaration (dummy, but required
in order to have make actually produce expected log files from the
`.depmod.log' suffix rule).
(TESTS): Add $(depmod_tests).
(EXTRA_DIST): Distribute depmod-tests.sh.
Other minor cosmetic changes and reorderings.
* bootstrap: Generate depmod-tests.am.
* tests/.gitignore: Updated.
Stefano Lattarini [Wed, 16 Feb 2011 12:46:33 +0000 (13:46 +0100)]
tests: avoid extra test generation (for `instspc' tests)
We don't really need to generate tons and tons of wrapper test
scripts (presently, ~ 90) just to run what is basically a single
test (specifically, `instspc-tests.sh') with different parameters.
In fact, the possibility of running a single driver script with
multiple files/configurations/parameters is one of the major
selling points of the 'parallel-tests' testsuite driver of
automake. So why not use ourselves more extensively the best
features we provide?
* tests/instspc-tests.sh: Update heading comments. Expect
to be executed, not sourced. Update handling of command-line
arguments. Adjust to reflect the renaming of "instspc tests"
from `instspc-*.test' to `*.instspc'.
* tests/instspc-data.test: Update heading comments. Execute
driver script `instspc-tests.sh', instead of sourcing it.
Give more informative messages in case of internal errors.
* tests/Makefile.am (TESTS_EXTENSIONS): Define explicitly (to
`.test' and `.instspc').
(TESTS): Add `$(instspc_tests)'.
(generated_tests): Remove `$(instspc_tests)'.
(INSTSPC_LOG_COMPILER): Define. Calls `instspc-tests.sh', so
that the "instspc tests" will be now executed by passing that
driver script a proper parameter.
($(instspc_tests)): Remove, we don't need anymore to generate
this tests.
($(instspc_tests:.test=.log)): Remove, substituted by ...
($(instspc_tests:.instspc=.log)): ... this.
($(instspc_tests)): New dependency declaration (dummy, but
required in order to have make actually produce expected log
files from the `.instspc.log' suffix rule).
(MAINTAINERCLEANFILES, generated_tests): Don't extend with
$(instspc_tests) anymore.
Update comments.
Stefano Lattarini [Wed, 16 Feb 2011 12:46:33 +0000 (13:46 +0100)]
build: improve the definition of the list of testcases
* tests/Makefile.am (handwritten_tests): New variable.
(generated_tests): Likewise.
(TESTS): Redefine as the union of the above.
(EXTRA_DIST): Extend using $(handwritten_tests) and
$(generated_tests) rather than $(TESTS).
* tests/gen-parallel-tests: Update accordingly, and
make more robust.