Stefano Lattarini [Tue, 31 May 2011 19:42:16 +0000 (21:42 +0200)]
testsuite: avoid generating `*-p.test' tests, use a wrapper script
The generated `*-p.test' tests had already become just thin layers
around the corresponding test scripts. This change makes the final
step, converting to the use of a generic wrapper script and thus
avoiding the extra test generation (similarly to what is done for
the `*.instspc' and `*.depmod' tests).
* tests/parallel-tests.sh: New file, driver script to run checks
the on the `parallel-tests' semantics by wrapping tests that use
the generic "Simple Tests" driver.
* tests/gen-parallel-tests: Update, mostly to reflect the new
`.ptest' extensions used for tests in $(parallel_tests).
* Makefile.am (TESTS_EXTENSIONS): Add `.ptest'.
(PTEST_LOG_COMPILER): Define, it calls `parallel-tests.sh'.
($(parallel_tests)): Do not really generate `*-p.test' tests
anymore; this is now just a dummy dependency declaration required
in order to have make actually produce expected log files from
the `.ptest.log' suffix rule.
(EXTRA_DIST): Distribute `parallel-tests.sh'.
(MAINTAINERCLEANFILES): Don't remove the `$(parallel_tests)', it
is not necessary anymore.
(generated_tests): Variable definition removed.
(TESTS): Update, by listing `$(parallel_tests)' directly instead
of `$(generated_tests)'
(expected_list_of_tests): Remove `$(generated_tests)'.
(maintainer-check-list-of-tests): No need to explicitly depend on
`$(expected_list_of_tests)' anymore.
Stefano Lattarini [Tue, 31 May 2011 16:07:56 +0000 (18:07 +0200)]
Merge branch 'master' into testsuite-work
* master:
automake, aclocal: honour configure-time AUTOCONF and AUTOM4TE
build: the user can override AUTOM4TE, AUTORECONF and AUTOUPDATE too
tests/README: fix example about `make -e' usage
Stefano Lattarini [Tue, 31 May 2011 16:03:48 +0000 (18:03 +0200)]
Merge branch 'maint'
* maint:
automake, aclocal: honour configure-time AUTOCONF and AUTOM4TE
build: the user can override AUTOM4TE, AUTORECONF and AUTOUPDATE too
tests/README: fix example about `make -e' usage
Stefano Lattarini [Sun, 29 May 2011 08:42:00 +0000 (10:42 +0200)]
automake, aclocal: honour configure-time AUTOCONF and AUTOM4TE
Currently, the Automake's own configure script allow definition
of AUTOCONF and AUTOM4TE, expected to point respectively to an
autoconf and autom4te programs. But while these definitions are
honoured in the Automake's build systems and test suite, they
were *not* honoured in the generated `automake' and `aclocal'
scripts. This behaviour, apart from being wrong in that it does
not allow the user enough freedom in choosing his tools, also
caused inconsistencies in the test suite, brining to spurious
failures.
Problem reported by Graham Reitz on the automake list; see thread:
<http://lists.gnu.org/archive/html/automake/2011-05/msg00022.html>
* automake.in ($traces): Use `@am_AUTOCONF', not simply `autoconf'.
* aclocal.in ($traces): Use `@am_AUTOM4TE', not simply `autom4te'.
* Makefile.am (do_subst): Substitute also `@am_AUTOCONF' and
`@am_AUTOM4TE'.
* NEWS: Update.
* THANKS: Update.
Stefano Lattarini [Sun, 29 May 2011 09:04:08 +0000 (11:04 +0200)]
build: the user can override AUTOM4TE, AUTORECONF and AUTOUPDATE too
Our build system allows the user to override AUTOCONF and AUTOHEADER
at configure time, and honours these overrides in our testsuite.
But it didn't do the same with AUTOM4TE, AUTORECONF and AUTOUPDATE.
This change fixes that inconsistency.
* configure.ac (am_AUTOM4TE, am_AUTOUPDATE, am_AUTORECONF): New
AC_SUBSTitutions. Update comments.
* tests/defs.in ($AUTOUPDATE): Default to `@am_AUTOUPDATE@' now.
($AUTOM4TE): New variable, defaulting to `@am_AUTOM4TE@'.
($AUTORECONF): New variable, defaulting to `@am_AUTORECONF@'.
* doc/Makefile.am ($(srcdir)/amhello-1.0.tar.gz): Update.
Stefano Lattarini [Sun, 29 May 2011 16:13:11 +0000 (18:13 +0200)]
tests: don't require gfortran if any fortran compiler is enough
* tests/silentf90.test ($required): Require `fortran', not
`gfortran'.
* tests/silentf77.test ($required): Require `fortran77', not
`gfortran'.
* tests/silent-many-generic.test ($required): Require `fortran'
and `fortran77' rather than `gfortran'.
Stefano Lattarini [Fri, 27 May 2011 13:05:40 +0000 (15:05 +0200)]
tests: improve `ccnoco*.test', better cross-compiling support
* tests/ccnoco3.test (Mycomp): Use the `$CC' chosen by `tests/defs'
instead of forcing `gcc' unconditionally. This ensures better
coverage in case of cross-compiling, when GCC can be named e.g.,
`i586-mingw32msvc-gcc', instead of simply `gcc'.
* tests/ccnoco.test: Likewise. Remove redundant checks. Modernize
the created `configure.in'. Run tests both in-tree and in VPATH.
Export `CC' to the overridden value only once.
* tests/ccnoco2.test: Slightly stricter grepping of automake
stderr. Add trailing `:' command.
Stefano Lattarini [Fri, 27 May 2011 08:22:31 +0000 (10:22 +0200)]
Merge branch 'master' into testsuite-work
Stefano Lattarini [Fri, 27 May 2011 08:20:25 +0000 (10:20 +0200)]
testsuite: each test case depends on `defs-static'
* tests/Makefile.am ($(TEST_LOGS)): Depends on `defs-static' too.
Simplify comments.
Stefano Lattarini [Fri, 3 Dec 2010 14:24:14 +0000 (15:24 +0100)]
testsuite: use $SHELL to run tests which are shell scripts
* tests/Makefile.am (TEST_LOG_COMPILER): Define so that the
configure-time $SHELL is used to run the tests.
* tests/defs: Add code to re-execute by default the running
test script with configure-time $SHELL. Updated comments.
* configure.ac: Check that `set -e' is working for $SHELL,
not for /bin/sh.
* tests/defs-static.in: Update comments.
* tests/README (Supported shells): Updated.
(Getting details from failures): Don't tell that tests are
run by /bin/sh by default.
Stefano Lattarini [Thu, 26 May 2011 20:28:45 +0000 (22:28 +0200)]
tests: split 'subst2.test' to improve modularity and coverage
* tests/subst2.test: Test removed, split into ...
* tests/programs-primary-rewritten.test: ... this test ...
* tests/subst-no-trailing-empty-line.test: ... and this one ...
* tests/extra-programs-empty.test: ... and this one.
* tests/Makefile.am (TESTS): Update.
Stefano Lattarini [Thu, 26 May 2011 13:42:02 +0000 (15:42 +0200)]
tests: tweak and improve tests on Automake conditionals
The "avoid the requirement of a working compiler" we refer about
below is obtained by disabling automatic dependency tracking and
defining `CC' (and `EXEEXT', `OBJEXT', `CXX', etc., if needed) to
dummy values, either directly in the `Makefile.am', or by AC_SUBST
in `configure.in'.
The "cosmetic changes" we refer about below consists in, e.g.,
adding a trailing `:' command to a script, using proper m4 quoting
in `configure.in' files, adding commands that offer more debugging
output, explicitly declaring phony targets as such in Makefiles,
making use of idioms more consistent with those employed in other
tests, following the GNU formatting standards more closely, etc.
* tests/cond.test: Cosmetic changes.
* tests/cond3.test: Likewise.
* tests/cond10.test: Likewise.
* tests/cond14.test: Likewise.
* tests/cond15.test: Likewise.
* tests/cond17.test: Likewise.
* tests/cond40.test: Likewise.
* tests/cond13.test: Likewise, and make grepping of the generated
`Makefile.in' slightly stricter.
* tests/cond6.test: Extend by also running ./configure, make and
"make install". Few cosmetic changes.
* tests/cond8.test: Likewise.
* tests/cond2.test: Do few cosmetic changes. Make grepping of
automake stderr stricter.
* tests/cond20.test: Likewise.
* tests/cond23.test: Likewise.
* tests/cond24.test: Likewise.
* tests/cond42.test: Likewise.
* tests/cond46.test: Likewise.
* tests/cond9.test: Move more checks in the `Makefile.am' instead
of grepping make's output.
* tests/cond38.test: Likewise.
* tests/cond11.test: Likewise. Avoid the requirement of a working
C compiler.
* tests/cond16.test: Likewise.
* tests/cond22.test: Likewise.
* tests/cond30.test: Likewise.
* tests/cond31.test: Likewise.
* tests/cond4.test: Likewise, and drop the now-useless GNU make
requirement as well.
* tests/cond18.test: Likewise.
* tests/cond19.test: Likewise.
* tests/cond22.test: Likewise.
* tests/cond25.test: Add trailing `:' command.
* tests/cond26.test: Likewise.
* tests/cond27.test: Likewise.
* tests/cond28.test: Likewise.
* tests/cond29.test: Use `unindent' function for better formatting.
Other minor cosmetic changes. Use a `timeout' program (if it's
available) to determine whether the script takes too long, instead
of just hoping that the user will notice an abnormally long test
execution time.
* tests/cond33.test: Extend a bit. Fix heading comments. Few more
cosmetic changes.
* tests/cond37.test: Ensure verbose printing of captured make
output. Minor cosmetic changes.
* tests/cond34.test: Likewise. Avoid the requirement of a working
C compiler. Make grepping of make output slightly stricter.
* tests/cond35.test: Quote literal dots in grep regexp. Add extra
debugging output. Minor cosmetic changes.
* tests/cond36.test: Likewise.
* tests/cond41.test: Do not simply check that aclocal fails, but
also grep its stderr for the expected error message.
* tests/cond43.test: Likewise (but the program being automake).
Few cosmetic changes.
* tests/condd.test: Avoid the requirement of a working C compiler.
* tests/condman3.test: Extend by using more man pages and more
manpage sections.
* tests/condman.test: Removed, it's completely superseded by
`condman3.test'.
* tests/Makefile.am (TESTS): Update.
Stefano Lattarini [Wed, 25 May 2011 17:05:15 +0000 (19:05 +0200)]
lex tests: avoid spurious failures when LEXLIB isn't found
The AC_PROG_LEX Autoconf macro does not diagnose a failure to find
the "lex library" expected to provide a `yywrap' function (function
which is required to link most lex-generated programs). On the
contrary, when all the link attempts (i.e., with `-ll' and `-lfl')
fail, configure declares that no lex library is needed, and simply
proceeds with the configuration process -- only for the build to
possibly fail later, at make time.
This behaviour might be (partly) intended; the Autoconf manual
reads:
``You are encouraged to use Flex in your sources, since it is
both more pleasant to use than plain Lex and the C source
it produces is portable. In order to ensure portability,
however, you must either provide a function `yywrap' or, if
you don't use it (e.g., your scanner has no `#include'-like
feature), simply include a `%noyywrap' statement in the
scanner's source.''
This AC_PROG_LEX behaviour is causing some spurious failures of the
Automake testsuite in environments which lack a proper library
providing `yywrap' (this happens for example in Linux->MinGW cross
compilations). But at this point is clear that a proper workaround
is to simply provide a fall-back implementation of `yywrap' in our
lexers.
* tests/cond35.test: Provide a dummy `yywrap' function.
* tests/lex3.test: Likewise.
* tests/lexvpath.test: Likewise.
* tests/silent-many-gcc.test: Likewise.
* tests/silent-many-generic.test: Likewise.
* tests/silent-lex-gcc.test: Likewise, and a dummy `main' too.
* tests/silent-lex-generic.test: Likewise.
* tests/lex-lib.test: New test.
* tests/lex-libobj.test: New test.
* tests/lex-nowrap.test: New test.
* tests/Makefile.am (TESTS): Update.
* THANKS: Update.
Thanks to Russ Allbery for the suggestion.
Stefano Lattarini [Wed, 25 May 2011 14:45:30 +0000 (16:45 +0200)]
testsuite: require C++ compiler explicitly in tests needing it
The list of the test scripts which needed a C++ compiler but
failed to require it explicitly has been found by running:
$ tests=`grep -lE 'CXX|\.c(c|\+\+|xx|pp)' *.test | tr '\n' ' '`
$ am_explicit_skips=yes CXX=false make check TESTS="$tests"
and looking for tests that reported FAIL instead of SKIP.
After the present change, all those previously failing tests
either pass or get correctly skipped.
* tests/subobj9.test: Require `c++', instead of explicitly
skipping on configure failure. Add excerpts and/or details from
the original bug report that prompted this tests to be written.
* tests/silentcxx-gcc.test: Require g++.
* tests/silentcxx.test: Require c++.
* tests/suffix3.test: Likewise.
Stefano Lattarini [Wed, 25 May 2011 14:15:27 +0000 (16:15 +0200)]
testsuite: don't require g++ where any C++ compiler is enough
* tests/specflg10.test ($required): Use `c++', not g++.
* tests/silent-many-generic.test: Likewise.
Stefano Lattarini [Mon, 23 May 2011 19:53:44 +0000 (21:53 +0200)]
testsuite: require C compiler explicitly in tests needing it
The list of the test scripts which needed a C compiler but
failed to require it explicitly has been found by running:
$ am_explicit_skips=yes CC=false make check
and looking for tests that reported FAIL instead of SKIP.
After the present change, all those previously failing tests
either pass or get correctly skipped.
* tests/aclocal4.test ($required): Add `cc'.
* tests/ansi.test: Likewise.
* tests/ansi10.test: Likewise.
* tests/ansi3.test: Likewise.
* tests/ansi3b.test: Likewise.
* tests/ansi5.test: Likewise.
* tests/ansi6.test: Likewise.
* tests/ansi7.test: Likewise.
* tests/ansi9.test: Likewise.
* tests/backcompat6.test: Likewise.
* tests/canon7.test: Likewise.
* tests/check5.test: Likewise.
* tests/check7.test: Likewise.
* tests/check8.test: Likewise.
* tests/cond4.test: Likewise.
* tests/cond11.test: Likewise.
* tests/cond16.test: Likewise.
* tests/cond18.test: Likewise.
* tests/cond19.test: Likewise.
* tests/cond22.test: Likewise.
* tests/cond30.test: Likewise.
* tests/cond31.test: Likewise.
* tests/cond32.test: Likewise.
* tests/cond34.test: Likewise.
* tests/cond35.test: Likewise.
* tests/cond39.test: Likewise.
* tests/condd.test: Likewise.
* tests/cygnus-dependency-tracking.test: Likewise.
* tests/depcomp2.test: Likewise.
* tests/depcomp6.test: Likewise.
* tests/depcomp7.test: Likewise.
* tests/depcomp8a.test: Likewise.
* tests/depcomp8b.test: Likewise.
* tests/depdist.test: Likewise.
* tests/depend2.test: Likewise.
* tests/depend5.test: Likewise.
* tests/distcleancheck.test: Likewise.
* tests/distname.test: Likewise.
* tests/exeext.test: Likewise.
* tests/exeext4.test: Likewise.
* tests/extradep.test: Likewise.
* tests/extradep2.test: Likewise.
* tests/gnits2.test: Likewise.
* tests/gnits3.test: Likewise.
* tests/instdir-ltlib.test: Likewise.
* tests/instdir-prog.test: Likewise.
* tests/instfail.test: Likewise.
* tests/instfail-libtool.test: Likewise.
* tests/lex3.test: Likewise.
* tests/lex5.test: Likewise.
* tests/lexvpath.test: Likewise.
* tests/lex-subobj-nodep.test: Likewise.
* tests/lflags.test: Likewise.
* tests/libobj-basic.test: Likewise.
* tests/libobj2.test: Likewise.
* tests/libobj7.test: Likewise.
* tests/libobj10.test: Likewise.
* tests/libobj16a.test: Likewise.
* tests/libobj16b.test: Likewise.
* tests/libobj17.test: Likewise.
* tests/libobj19.test: Likewise.
* tests/libtool2.test: Likewise.
* tests/libtool3.test: Likewise.
* tests/libtool7.test: Likewise.
* tests/libtool9.test: Likewise.
* tests/libtoo10.test: Likewise.
* tests/libtoo11.test: Likewise.
* tests/ltcond.test: Likewise.
* tests/ltcond2.test: Likewise.
* tests/ltconv.test: Likewise.
* tests/ltinit.test: Likewise.
* tests/ltlibsrc.test: Likewise.
* tests/ltorder.test: Likewise.
* tests/nobase.test: Likewise.
* tests/nobase-libtool.test: Likewise.
* tests/mmodely.test: Likewise.
* tests/parallel-tests5.test: Likewise.
* tests/parallel-tests-suffix-prog.test: Likewise.
* tests/parallel-tests-ext-driver-prog.test: Likewise.
* tests/posixsubst-ldadd.test: Likewise.
* tests/posixsubst-libraries.test: Likewise.
* tests/posixsubst-ltlibraries.test: Likewise.
* tests/posixsubst-programs.test: Likewise.
* tests/posixsubst-sources.test: Likewise.
* tests/pr87.test: Likewise.
* tests/pr204.test: Likewise.
* tests/pr224.test: Likewise.
* tests/pr300-lib.test: Likewise.
* tests/pr300-ltlib.test: Likewise.
* tests/pr300-prog.test: Likewise.
* tests/pr401.test: Likewise.
* tests/pr401b.test: Likewise.
* tests/pr401c.test: Likewise.
* tests/remake-gnulib-add-acsubst.test: Likewise.
* tests/remake-gnulib-add-header.test: Likewise.
* tests/regex.test: Likewise.
* tests/repeated-options.test: Likewise.
* tests/silent.test: Likewise.
* tests/silent3.test: Likewise.
* tests/silent9.test: Likewise.
* tests/silent-lex-generic.test: Likewise.
* tests/silent-many-generic.test: Likewise.
* tests/silent-yacc-generic.test: Likewise.
* tests/specflg7.test: Likewise.
* tests/specflg9.test: Likewise.
* tests/specflg10.test: Likewise.
* tests/stdinc.test: Likewise.
* tests/strip.test: Likewise.
* tests/strip2.test: Likewise.
* tests/strip3.test: Likewise.
* tests/subdirbuiltsources.test: Likewise.
* tests/subobj3.test: Likewise.
* tests/subobj6.test: Likewise.
* tests/subobj11a.test: Likewise.
* tests/subpkg.test: Likewise.
* tests/subst2.test: Likewise.
* tests/subst3.test: Likewise.
* tests/substref.test: Likewise.
* tests/substre2.test: Likewise.
* tests/suffix5.test: Likewise.
* tests/suffix8.test: Likewise.
* tests/suffix10.test: Likewise.
* tests/suffix11.test: Likewise.
* tests/suffix12.test: Likewise.
* tests/suffix13.test: Likewise.
* tests/target-cflags.test: Likewise.
* tests/transform.test: Likewise.
* tests/transform2.test: Likewise.
* tests/yacc-basic.test: Likewise.
* tests/yacc-d-basic.test: Likewise.
* tests/yacc-clean.test: Likewise.
* tests/yacc-dist-nobuild.test: Likewise.
* tests/yacc-nodist.test: Likewise.
* tests/yaccvpath.test: Likewise.
* tests/yacc-d-vpath.test: Likewise.
* tests/yacc4.test: Likewise.
* tests/yacc7.test: Likewise.
* tests/yacc8.test: Likewise.
* tests/yaccdry.test: Likewise.
* tests/yflags-cmdline-override.test: Likewise.
* tests/yflags-force-override.test: Likewise.
* tests/python-virtualenv.test: Likewise. Also, improve skip
messages.
* tests/subobj5.test ($required): Add `cc'.
(Makefile.am): Set `AUTOMAKE_OPTIONS' to `subdir-objects', and
add new checking rules `test-build' and `test-distdir'.
Extend the test by building and examining the distdir, the
program, and the object files.
* tests/postproc.test ($required): Add `cc'.
Avoid the explicit `|| Exit $?' after call to configure, which
is now either redundant (e.g., when `am_explicit_skips' is
unset), or counter-productive (e.g., when `am_explicit_skips'
is set to "yes").
* tests/pr243.test: Likewise.
* tests/pr266.test: Likewise.
* tests/pr220.test: Simplify so that it doesn't require a
C compiler anymore.
* tests/subdir5.test: Likewise.
* tests/subdir8.test: Likewise.
* tests/lflags.test: Likewise.
* tests/yflags.test: Likewise.
* tests/yflags-force-conditional.test: Likewise.
* tests/lflags2.test: Simplify so that it doesn't require a
C++ compiler anymore.
* tests/yflags2.test: Likewise.
* tests/autohdrdry.test (configure.in): Remove unneeded call
to `AC_PROG_CC'.
* tests/pr287.test: Likewise.
* tests/check6.test: Likewise.
* tests/cond21.test: Likewise, plus some cosmetic adjustments.
* tests/upc.test: Skip if configure fails with status `77'.
* tests/upc3.test: Likewise.
* tests/vala4.test: Likewise.
* tests/nostdinc.test: Likewise. Also, make grepping checks
on `Makefile.in'. Update heading comments.
* tests/compile5.test: When we must skip, skip explicitly,
and with a meaningful message.
* tests/instspc-tests.sh: When running in "test-build" or
"test-install" mode (as determined by the value of variable
`$instspc_action'), require `cc'. Adjust comments.
* tests/depmod-tests.sh: Likewise.
Stefano Lattarini [Mon, 23 May 2011 13:51:31 +0000 (15:51 +0200)]
testsuite: allow user to ask for stricter skip semantics
Given how the Automake testsuite is currently structured, if a
command in a test script fails with status `77', the global test
outcome is considered a SKIP, because the value of `77' for `$?'
is passed to the exit trap. Indeed, this happens in practice, as
an autoconf-generated configure script can exit with status `77'
if it fails to find, e.g., a required compiler.
While this behaviour is quite useful for avoiding spurious test
failures in the wild, it can also sometimes prevent the Automake
developers to easily see and declare the requirements of their
tests.
This change introduces a new variable `am_explicit_skips', meant to
be user-overridable, and which, when set to a "true" value (i.e.,
`yes' or `1'), require a direct call to `Exit 77' in order to make
the test outcome be considered a SKIP.
* tests/defs.in ($am__test_skipped): New variable, initialized
to `no'.
(Exit): Set `$am__test_skipped' to `yes' if passed an exit status
of 77.
(trap '...' 0): When `$am_explicit_skips' is set to a "true" value,
reset an exit status of `77' to `78' if $am__test_skipped is not
set to `yes'.
* tests/self-check-exit.test: Adjust: unset `am_explicit_skips'.
* tests/self-check-explicit-skips.test: New test.
* tests/Makefile.am (TESTS): Update.
Stefano Lattarini [Tue, 24 May 2011 08:42:27 +0000 (10:42 +0200)]
testsuite: user can force skipping of compiler-requiring tests
* tests/defs (cc, c++, fortran, fortran77): Skip the test if
the relevant compiler is disabled by having the corresponding
variable (CC, CXX, FC and F77, respectively) set to "false".
(yacc): For consistency, skip the test when the YACC variable
is set to "false", not when it's set to "no". Since we are at
it, fix the skip message to be shorter and more consistent.
Stefano Lattarini [Mon, 23 May 2011 12:26:58 +0000 (14:26 +0200)]
tests: tweak and improve tests on "Simple Tests" driver
* tests/check.test: Make grepping of generated Makefile.in
slightly stricter. Add trailing `:' command.
* tests/check2.test: Ensure verbose printing of captured make
output. Make grepping of generated Makefile.in and of make
output stricter. Add trailing `:' command.
* tests/check3.test: Likewise. Also, fix minor error in a
comment.
* tests/check4.test: Stricter grepping of make output. Use
`using_gmake' subroutine instead of copy&paste from defs.in
to determine whether $MAKE is GNU make. Add a trailing `:'
command.
* tests/check6.test: Add trailing `:' command.
* tests/check10.test: Prefer redirection `>' over `>>' when
creating new files. Ensure verbose printing of captured make
output.
* tests/check11.test: Likewise.
* tests/check7.test: Likewise. Make grepping of make output
stricter. Prefer "make distcheck" over a simple "make check"
in a VPATH build. Add trailing `:' command.
(a.c): Format following GNU coding standards.
(Makefile.am): Explicitly declare target `print-xfail-tests'
as PHONY. Distribute $(check_SCRIPTS), for distcheck.
* tests/parallel-tests.test: Prefer redirection `>' over `>>'
when creating new files.
* tests/parallel-tests2.test: Likewise.
* tests/parallel-tests9.test: Likewise.
* tests/parallel-tests8.test: Likewise. Make grepping of
automake stderr stricter.
* tests/check8.test: Likewise. Make formatting of created
*.c files more consistent with GNU Coding Standards.
* tests/check-subst-prog.test (configure.in): Break overly
long lines.
(pass-prog.c, xfail-prog.c): Format them more consistently
with GNU Coding Standards.
* tests/parallel-tests3.test: Use a shorter and clearer skip
message. Use the `unindent' subroutine to improve readability
of indented loop bodies. Simplify the hack used to please
maintainer-check.
* tests/parallel-tests5.test: Make formatting of created *.c
files more consistent with GNU Coding Standards. Use the
`unindent' subroutine to improve readability of indented loop
bodies.
* tests/parallel-tests10.test: Add blank line, for clarity.
Prefer trailing `:' over trailing `Exit 0', for consistency
with other tests.
Ralf Wildenhues [Sun, 22 May 2011 17:02:27 +0000 (19:02 +0200)]
tests/README: fix example about `make -e' usage
* tests/README (Section "Writing test cases" subsection "Do"): When
some variable is never initialized in the Makefile, `-e' is not
necessary in order to override it. DESTDIR is such a variable: we
ensure that we do not ever initialize it. And as such, it is quite
portable to use:
$ make DESTDIR=/foo/bar install
and in fact, quite widely used.
So our example about when `make -e' is required, which references
the `DESTDIR' variable, is poorly chosen, if not downright wrong.
Rewrite it to use `prefix' as the overridden variable instead.
Stefano Lattarini [Sat, 21 May 2011 09:03:00 +0000 (11:03 +0200)]
tests/README: small consistency improvement
* tests/README (Section "User interface" subsection "Running the
tests"): In examples, be more consistent w.r.t. the differences
between GNU and non-GNU make.
Stefano Lattarini [Sat, 21 May 2011 08:59:53 +0000 (10:59 +0200)]
tests/README: suggest how to run tests in cross-compile mode
* tests/README (Section "User interface" subsection "Running the
tests"): Briefly explain how to override 'host_alias' at runtime
to force the use of cross-compilers by the testsuite. Give an
example.
Stefano Lattarini [Sat, 21 May 2011 07:12:06 +0000 (09:12 +0200)]
test defs: better detection of cross-compile mode
* configure.ac (AC_CANONICAL_HOST): New, probably not strictly
necessary, but useful to complements AC_CANONICAL_BUILD.
* tests/defs-static.in ($host_alias, $build_alias): New variables,
exported, user-overridable, defaults AC_SUBST'd by configure.
* tests/defs (cross_compiling): Consider cross-compilation mode
active only if `$host_alias' is not empty *and* it differs from
`$build_alias'. This is more faithful to the way configure
determines whether it is cross-compiling, but still allows the
user to easily force cross-compilation by overriding `host_alias'
and `build_alias' at runtime.
From a report by Ralf Wildenhues.
Stefano Lattarini [Fri, 20 May 2011 22:08:02 +0000 (00:08 +0200)]
Merge branch 'master' into testsuite-work
* master:
testsuite: avoid re-running few tests with 'parallel-tests' option
testsuite: allow tests to avoid the use of 'parallel-tests' option
tests/README: update obsoleted advice
Stefano Lattarini [Fri, 20 May 2011 21:33:09 +0000 (23:33 +0200)]
Merge branch 'maint'
* maint:
testsuite: avoid re-running few tests with 'parallel-tests' option
testsuite: allow tests to avoid the use of 'parallel-tests' option
tests/README: update obsoleted advice
Stefano Lattarini [Fri, 20 May 2011 21:18:39 +0000 (23:18 +0200)]
testsuite: avoid re-running few tests with 'parallel-tests' option
Some tests in our testsuite use the 'simple-tests' driver only
marginally, or simply as a mean to conveniently check unrelated
invariants. It makes little sense to force these tests to also
run with the 'parallel-tests' Automake option active, as doing so
offers no real gain in coverage, while often causing a measurable
overhead in execution time (for an already too-slow testsuite).
* tests/pr401.test (parallel_tests): Define to "no", to prevent
the generation of a sibling test script using the 'parallel-tests'
driver.
* tests/pr401b.test: Likewise.
* tests/pr401c.test: Likewise.
Stefano Lattarini [Fri, 20 May 2011 20:31:40 +0000 (22:31 +0200)]
testsuite: allow tests to avoid the use of 'parallel-tests' option
* tests/gen-parallel-tests: Do not generate "siblings" for tests
that explicitly define the `parallel_tests', whether to "yes" or
to any other value. Extend heading comments to give a rationale
for this behaviour.
* tests/README: Update.
Stefano Lattarini [Fri, 20 May 2011 20:15:32 +0000 (22:15 +0200)]
tests/README: update obsoleted advice
* tests/README (Section "Writing test cases" subsection "Do"):
Do not suggest to use the `*-p.test' pattern for the names of
hand-written tests which use the `parallel-tests' Automake option.
Not only is this not respected by the existing tests, but it is
more likely to cause conflicts with auto-generated tests.
So, suggest to *avoid* using the `*-p.test' pattern in names
of hand-written tests instead.
(Section "Writing test cases" subsection "Do not"): When
suggesting not to override Makefile variables using command
line arguments, do not use the badly outdated variables `U'
and 'ANSI2KNR' in the example; instead, use the more common
and typical `DESTDIR'.
Stefano Lattarini [Thu, 19 May 2011 20:50:35 +0000 (22:50 +0200)]
fix typos in latest ChangeLog entry
Stefano Lattarini [Sun, 15 May 2011 14:33:01 +0000 (16:33 +0200)]
tests: avoid spurious failures in cross-compile mode
* tests/depcomp2.test: Ensure verbose printing of captured stderr
from configure.
* tests/ansi3.test ($required): Add 'native', as the ansi2knr
functionality is not meant to work with a cross-compiler.
* tests/ansi3b.test: Likewise.
* tests/ansi5.test: Likewise.
* tests/ansi6.test: Likewise.
* tests/ansi7.test: Likewise.
* tests/ansi8.test: Likewise.
* tests/ansi10.test: Likewise.
* tests/subobj3.test: Likewise.
* tests/check8.test ($required): Add 'native', as this test is
not meant to work with a cross-compiler.
* tests/gnits2.test: Likewise.
* tests/ltconv.test: Likewise.
* tests/ltcond2.test: Likewise.
* tests/parallel-tests5.test: Likewise.
* tests/specflg7.test: Likewise.
* tests/specflg8.test: Likewise.
* tests/check5.test: Likewise. Also, ensure verbose printing of
captured make output, and thrown in few cosmetic and consistency
improvements.
* tests/ltinit.test: Be laxer in grepping configure output, to
avoid spurious failures on systems which lack POSIX dynamic
linking (e.g., MinGW), or when cross-compiling for such systems.
* tests/ansi9.test: Skip parts of the test that would require a
native compiler.
* tests/backcompat6.test: Likewise.
* tests/depcomp8a.test: Likewise.
* tests/pr401.test: Likewise.
* tests/pr401b.test: Likewise.
* tests/pr401c.test: Likewise.
* tests/target-cflags.test: Likewise.
* tests/lex3.test: Likewise.
* tests/yacc-basic.test: Likewise.
* tests/autohdr4.test: Likewise. Also, extend the test to ensure
more coverage in cross-compile mode.
* tests/canon7.test: Likewise.
* tests/check9.test: Removed, split into two new tests ...
* tests/check-subst-prog.test: ... this one, which requires a
native C compiler ...
* tests/check-subst.test: ... and this one, which doesn't require
any compiler.
* tests/parallel-tests4.test: Removed, split into two new tests ...
* tests/parallel-tests-suffix-prog.test: ... this one, which
requires a native C compiler ...
* tests/parallel-tests-suffix.test: ... and this one, which
doesn't require any compiler.
* tests/parallel-tests7.test: Removed, split into two new tests ...
* tests/parallel-tests-ext-driver-prog.test: ... this one, which
requires a native C compiler ...
* tests/parallel-tests-ext-driver.test: ... and this one, which
doesn't require any compiler.
* tests/Makefile.am (TESTS): Update.
Stefano Lattarini [Thu, 19 May 2011 20:07:48 +0000 (22:07 +0200)]
Merge branch 'master' into testsuite-work
* master:
test defs: rename requirement 'non-cross' -> 'native'
Stefano Lattarini [Thu, 19 May 2011 19:53:02 +0000 (21:53 +0200)]
Merge branch 'maint'
* maint:
test defs: rename requirement 'non-cross' -> 'native'
Stefano Lattarini [Thu, 19 May 2011 19:48:27 +0000 (21:48 +0200)]
test defs: rename requirement 'non-cross' -> 'native'
* tests/defs.in (non-cross): Rename requirement ...
(native): ... to this, which is clearer and fits the
existing lingo better.
Suggestion by Ralf Wildenhues.
Stefano Lattarini [Sun, 15 May 2011 13:38:14 +0000 (15:38 +0200)]
Merge branch 'master' into testsuite-work
* master:
testsuite: be more cross-compile friendly
tests: fix portability issues in 'repeated-options.test'
Stefano Lattarini [Sun, 15 May 2011 13:35:27 +0000 (15:35 +0200)]
Merge branch 'maint'
* maint:
testsuite: be more cross-compile friendly
Stefano Lattarini [Sun, 15 May 2011 12:30:43 +0000 (14:30 +0200)]
testsuite: be more cross-compile friendly
* tests/defs.in (cross_compiling): New subroutine.
(am__tool_prefix): New internal variable.
(gcc, g++, gcj): Force the use of the correct "tool prefix"
when cross compiling.
(gfortran, g77, non-cross): New requirements.
Stefano Lattarini [Sun, 15 May 2011 12:41:57 +0000 (14:41 +0200)]
Merge branch 'test-repeated-options-portability-fix'
Stefano Lattarini [Sun, 15 May 2011 12:41:35 +0000 (14:41 +0200)]
tests: fix portability issues in 'repeated-options.test'
* tests/repeated-options.test: Do not assume that object files
have `.o' suffix and executables have no default suffix; instead,
use `$(OBJEXT)' and `$(EXEEXT)'.
Stefano Lattarini [Sat, 7 May 2011 13:40:54 +0000 (15:40 +0200)]
Merge branch 'master' into testsuite-work
* master:
tests: fix spurious failure of extradep.test on FreeBSD
Stefano Lattarini [Sat, 7 May 2011 13:39:37 +0000 (15:39 +0200)]
Merge branch 'maint'
* maint:
tests: fix spurious failure of txinfo21.test on FreeBSD
Stefano Lattarini [Sat, 7 May 2011 13:38:56 +0000 (15:38 +0200)]
Merge branch 'maint' into testsuite-work
* maint:
tests: fix spurious failure of txinfo21.test on FreeBSD
Stefano Lattarini [Sat, 7 May 2011 13:35:35 +0000 (15:35 +0200)]
Merge branch 'test-silentcxx-gcc-fix' into testsuite-work
Stefano Lattarini [Sat, 7 May 2011 12:18:47 +0000 (14:18 +0200)]
tests: fix syntax error in 'silentcxx-gcc.test'
* tests/silentcxx-gcc.test (for config_args in ...): Remove extra
trailing backslash from the list of looped-on items.
Stefano Lattarini [Sat, 7 May 2011 13:23:57 +0000 (15:23 +0200)]
Merge branch 'test-extradep-fix'
Stefano Lattarini [Sat, 7 May 2011 13:23:28 +0000 (15:23 +0200)]
typofix in ChangeLog
Stefano Lattarini [Sat, 7 May 2011 13:10:14 +0000 (15:10 +0200)]
Merge branch 'test-extradep-fix'
Stefano Lattarini [Sat, 7 May 2011 10:58:51 +0000 (12:58 +0200)]
tests: fix spurious failure of extradep.test on FreeBSD
* tests/extradep.test: When using `ls -t' to determine whether a
file has been updated, make sure to use as reference a file whose
timestamp is expected to be *strictly* older that that of the file
being checked. This is required because at least FreeBSD `ls' do
not sort files with the same timestamp in alphabetical order when
using the `-t' option.
* tests/extradep2.test: Likewise.
Stefano Lattarini [Sat, 7 May 2011 12:34:39 +0000 (14:34 +0200)]
tests: fix spurious failure of txinfo21.test on FreeBSD
* tests/txinfo21.test: Use the `is_newest' subroutine instead of
the `ls -t' hack to to determine whether a file has been updated.
This is required because at least FreeBSD `ls' do not sort files
with the same timestamp in alphabetical order when using the `-t'
option.
Stefano Lattarini [Sat, 7 May 2011 09:50:02 +0000 (11:50 +0200)]
Merge branch 'fix-depmod-tests-generation' into testsuite-work
Stefano Lattarini [Sat, 7 May 2011 09:49:40 +0000 (11:49 +0200)]
depmod tests: fix bug in depmodes list extraction
* tests/depmod-tests.sh (get_depmodes): Use `echo $all_depmodes'
instead of `... | tr "$nl" "$sp"' to strip extra whitespaces and
newlines from `$all_depmodes'. The latter idiom with `tr' is
wrong since it gets run before `./defs' is sourced, and thus `$nl'
and `$sp' are undefined.
Bug revealad by FreeBSD tr(1) implementation, which doesn't accept
empty strings as arguments.
Stefano Lattarini [Wed, 4 May 2011 21:36:25 +0000 (23:36 +0200)]
tests: add excerpts from few original bug reports, for clarity.
* tests/comment6.test: Add excerpts and/or details from the
original bug report that prompted this tests to be added and/or
extended.
* tests/dejagnu4.test: Likewise.
* tests/installdir.test: Likewise.
Stefano Lattarini [Wed, 4 May 2011 21:09:54 +0000 (23:09 +0200)]
Merge branch 'master' into testsuite-work
Stefano Lattarini [Wed, 4 May 2011 21:06:16 +0000 (23:06 +0200)]
ChangeLog: some typofixes
Stefano Lattarini [Wed, 4 May 2011 20:50:59 +0000 (22:50 +0200)]
Merge branch 'maint'
Stefano Lattarini [Wed, 4 May 2011 20:42:19 +0000 (22:42 +0200)]
ChangeLog: various typofixes (and related reformatting)
Stefano Lattarini [Wed, 4 May 2011 20:21:53 +0000 (22:21 +0200)]
Merge branch 'maint'
With some edits to `tests/defs' and `ChangeLog', to reflect the
fact that in master we already had a `yacc' requirement, and no
more had an explicitly-recognized `bison' requirement.
Stefano Lattarini [Wed, 4 May 2011 16:42:11 +0000 (18:42 +0200)]
tests defs: allow requirements for compilers (mostly dummy)
Most of the new requirements that are now accepted in `$required'
as consequence of this patch are still dummy. They are planned
to be implemented only in master (or in some derived branch), but
having them here (even just as no-op) will allow for an easier
integration/backporting of potential new testcases.
* tests/defs.in (cc, c++, fortran, fortran77): New requirements,
still dummy.
(flex): New requirement, picking LEX for configure.
(lex): New requirement, alias for `flex'. A more appropriate
implementation, looking for a generic `lex' program, will follow
in the future.
(yacc): New requirement, alias for `bison'. A more appropriate
implementation, looking for a generic `yacc' program, will follow
in the future.
Stefano Lattarini [Wed, 4 May 2011 09:58:21 +0000 (11:58 +0200)]
tests: 'silent-many-gcc.test' improved and made more robust
* tests/silent-many-gcc.test: Also force "fast" gcc depmode
for C++ compilations. Add sanity checks verifying that the
cache variables we force are really used by configure. Fix
typo in comments.
Stefano Lattarini [Wed, 4 May 2011 09:48:19 +0000 (11:48 +0200)]
tests: drop useless requirements of 'g++'
* tests/lflags2.test ($required): Remove 'g++', as any working
C++ compiler should be enough, and ./configure will cause the
test to skip if no working C++ compiler is found.
* tests/yflags2.test: Likewise.
* tests/subobj9.test: Likewise.
* tests/silentcxx.test: Likewise. Also, do not force depmodes
that could cause non-GNU C++ compilers to fail.
* tests/silentcxx-gcc.test: New test, like `silentcxx.test',
but forcing "fast" gcc depmode (and thus requiring the GNU C++
compiler).
* tests/specflg10.test: Add proper "fixme" comment telling that
we should make this test work with a generic C++ compiler.
* tests/Makefile.am (TESTS): Update.
Stefano Lattarini [Tue, 3 May 2011 21:40:00 +0000 (23:40 +0200)]
tests defs: one more environment sanitization (corner case)
* tests/defs (am__using_gmake): Initialize. The `using_gmake'
subroutine was using this variable for caching, but wasn't
initializing it, which could cause problems in the (admittedly
very unlikely) case in which it was pre-existent in the
environment.
Stefano Lattarini [Tue, 3 May 2011 17:23:31 +0000 (19:23 +0200)]
Merge branch 'master' into testsuite-work
Stefano Lattarini [Tue, 3 May 2011 17:21:28 +0000 (19:21 +0200)]
Merge branch 'tests-man-fixes'
Stefano Lattarini [Tue, 3 May 2011 17:19:13 +0000 (19:19 +0200)]
tests: add forgotten test scripts to $(TESTS)
* tests/Makefile.am (TESTS): Added test scripts man[678].test,
which have been present on the filesystem and in the repository,
but which had erroneously been left out from $(TESTS).
Stefano Lattarini [Tue, 3 May 2011 17:08:30 +0000 (19:08 +0200)]
Merge branch 'check-tests-list' into testsuite-work
Stefano Lattarini [Sat, 30 Jan 2010 00:23:44 +0000 (01:23 +0100)]
maintcheck: consistency of list of test scripts
* CheckListOfTests.am: New file.
(maintainer-check-list-of-tests): New target,
check for consistency between list of tests defined in the
including Makefile and list of tests on the filesystem.
(clean-maintcheck-testslist-tmp): New rule, to clean up the
temporary files that might be left around by the rules associated
with the previous target.
(clean-local): Depend on it.
* lib/Automake/tests/Makefile.am: Include `CheckListOfTests.am'.
(maintainer-check): Added dependency from the new target
`maintainer-check-list-of-tests'.
* tests/Makefile.am: Likewise.
* Makefile.am (maintainer-check-list-of-test): New target,
calling recursively into `tests/' and `lib/Automake/tests/',
using ...
(TEST_SUBDIRS): ... this new variable.
Stefano Lattarini [Tue, 3 May 2011 16:24:29 +0000 (18:24 +0200)]
tests: ensure verbose printing of captured make output
* tests/libtool7.test: Ensure verbose printing of captured
make output.
* tests/libtool9.test: Likewise.
Stefano Lattarini [Tue, 3 May 2011 15:51:42 +0000 (17:51 +0200)]
tests: drop useless requirements of gcc (plus testsuite tweakings)
* tests/libobj14.test ($required): Remove 'gcc', as we never run
the C compiler.
* tests/targetclash.test: Likewise.
* tests/ansi6.test ($required): Remove 'gcc', as any working
C compiler should be enough.
* tests/ansi7.test: Likewise.
* tests/ansi9.test: Likewise.
* tests/ansi10.test: Likewise.
* tests/lex5.test: Likewise.
* tests/lexvpath.test: Likewise.
* tests/mmodely.test: Likewise.
* tests/pr204.test: Likewise.
* tests/pr300-lib.test: Likewise.
* tests/pr300-prog.test: Likewise.
* tests/primary3.test: Likewise.
* tests/specflg7.test: Likewise.
* tests/specflg8.test: Likewise.
* tests/subdir5.test: Likewise.
* tests/subdir8.test: Likewise.
* tests/subobj6.test: Likewise.
* tests/subst3.test: Likewise.
* tests/substre2.test: Likewise.
* tests/yacc6.test: Likewise.
* tests/yacc8.test: Likewise.
* tests/depcomp2.test: Likewise. Also, avoid clobbering user-set
CFLAGS.
* tests/lex3.test: Likewise.
* tests/ansi3.test: Likewise. Also, avoid 'CC=gcc' in configure.
* tests/ansi3b.test: Likewise.
* tests/ansi5.test: Likewise.
* tests/autohdr4.test ($required): Remove 'gcc', as any working
C compiler should be enough.
Also, do not reject slow dependency extractors (which we might
be forced to use now that $CC is not necessarily gcc anymore).
* tests/cond16.test ($required): Remove 'gcc', as any working
C compiler should be enough.
Since we are at it, throw in few minor tweakings (mostly cosmetic,
stylistic, or consistency-related).
* tests/cond18.test: Likewise.
* tests/cond35.test: Likewise.
* tests/gnits2.test: Likewise.
* tests/libtool3.test: Likewise.
* tests/libtool7.test: Likewise.
* tests/libtool9.test: Likewise.
* tests/ltcond.test: Likewise.
* tests/ltcond2.test: Likewise.
* tests/ltconv.test: Likewise.
* tests/ltlibsrc.test: Likewise.
* tests/nobase.test: Likewise.
* tests/nobase-libtool.test: Likewise.
* tests/pr220.test: Likewise.
* tests/pr224.test: Likewise.
* tests/pr300-ltlib.test: Likewise.
* tests/pr401.test: Likewise.
* tests/pr401b.test: Likewise.
* tests/pr401c.test: Likewise.
* tests/subpkg.test: Likewise.
* tests/target-cflags.test: Likewise.
* tests/transform.test: Likewise.
* tests/yacc4.test: Likewise.
* tests/cond19.test: Likewise. Also, avoid clobbering user-set
CFLAGS.
* tests/cond4.test: Likewise.
* tests/depend2.test: Likewise.
* tests/pr87.test: Likewise.
* tests/subobj3.test: Likewise.
* tests/substref.test: Likewise.
Stefano Lattarini [Tue, 3 May 2011 10:00:10 +0000 (12:00 +0200)]
test defs: increase coverage w.r.t. GNU make
* tests/defs (GNUmake): Instead of just skipping the tests
requiring GNU make if $MAKE is not GNU make, try to look for
it and, if found, redefine $MAKE accordingly. This will help
to transparently increase coverage on non-GNU systems which
have GNU make available in PATH.
Stefano Lattarini [Tue, 3 May 2011 09:29:44 +0000 (11:29 +0200)]
tests: improve few inter-tests references
* tests/acloca22.test: Improve and extend the heading comments.
Add reference to related tests 'remake-deleted-m4-file.test' and
'remake-renamed-m4-macro-and-file.test'. Since we are at it,
add trailing `:' command, and few blank lines for readability.
* tests/remake-renamed-m4-macro-and-file.test: In the heading
comments, add reference to the related test 'acloca22.test'.
Stefano Lattarini [Tue, 3 May 2011 09:06:30 +0000 (11:06 +0200)]
Merge branch 'master' into testsuite-work
Stefano Lattarini [Mon, 2 May 2011 17:04:01 +0000 (19:04 +0200)]
tests: few more checks on automatic remake support
* tests/remake-all-1.test: New test, check that the "all" target
triggers rebuilt of outdated Makefiles.
* tests/remake-all-2.test: Likewise, but for when the makefiles
are not named `Makefile'.
* tests/Makefile.am (TESTS): Update.
Stefano Lattarini [Mon, 2 May 2011 16:05:34 +0000 (18:05 +0200)]
tests: check some remake features with non-GNU make too
* tests/acloca14.test ($required): Drop "GNUmake". This test
works as-is with non-GNU make implementations.
* tests/remake1a.test: Likewise.
* tests/aclocal4.test ($required): Drop "GNUmake".
Modify the test to have it work also with non-GNU make.
* tests/remake5.test: Likewise.
* tests/remake8a.test: Likewise.
* tests/remake8b.test: Likewise.
* tests/remake9a.test: Likewise.
* tests/remake9b.test: Likewise.
* tests/remake9c.test: Likewise.
* tests/remake9d.test: Likewise.
* tests/remake10a.test: Likewise.
* tests/remake10b.test: Likewise.
* tests/remake10c.test: Likewise.
* tests/remake12.test: ($required): Drop "GNUmake".
Adapt the test to make it work also with non-GNU make (if
it supports an "include" directive).
Stefano Lattarini [Mon, 2 May 2011 22:27:07 +0000 (00:27 +0200)]
tests: fix spurious failure in 'color2.test'
* tests/color2.test: If $MAKE contains command-line arguments (as
in e.g., "make -j2"), expect's directive "spawn $env(MAKE)" fails
spuriously, because it tries to run "$MAKE" as a single command.
Fix this with proper uses of the TCL `eval' builtin.
Stefano Lattarini [Sun, 17 Apr 2011 12:28:17 +0000 (14:28 +0200)]
tests: extend tests on dependency tracking with VPATH
* tests/depmod-tests.sh: New script, fulfilling a threefold role:
1. it is called to generate a Makefile.am snippet, containing the
definition of proper lists of tests;
2. it is called to set up a directory containing some common data
files and autotools-generated files used by the aforementioned
tests (this is done for speed reasons only); and
3. it is called to properly run those tests, one at a time.
* tests/depcomp9.test: Delete, it's obsolete now.
* tests/depcomp10.test: Likewise.
* tests/Makefile.am ($(srcdir)/depmod-tests.am): Include this
snippet, which defines ...
(depmode_tests): ... this new macro, containing the list of the
newly generated `*.depmod' tests.
(TESTS_EXTENSIONS): Add `.depmod'.
(DEPMOD_LOG_COMPILER): Define. It calls `depmod-tests.sh', so that
the "depmode tests" will be executed by passing that driver script
a proper parameter.
($(depmod_tests)): New dependency declaration (dummy, but required
in order to have make actually produce expected log files from the
`.depmod.log' suffix rule).
(TESTS): Add $(depmod_tests).
(EXTRA_DIST): Distribute depmod-tests.sh.
Other minor cosmetic changes and reorderings.
* bootstrap: Generate depmod-tests.am.
* tests/.gitignore: Updated.
Stefano Lattarini [Wed, 16 Feb 2011 12:46:33 +0000 (13:46 +0100)]
tests: avoid extra test generation (for `instspc' tests)
We don't really need to generate tons and tons of wrapper test
scripts (presently, ~ 90) just to run what is basically a single
test (specifically, `instspc-tests.sh') with different parameters.
In fact, the possibility of running a single driver script with
multiple files/configurations/parameters is one of the major
selling points of the 'parallel-tests' testsuite driver of
automake. So why not use ourselves more extensively the best
features we provide?
* tests/instspc-tests.sh: Update heading comments. Expect
to be executed, not sourced. Update handling of command-line
arguments. Adjust to reflect the renaming of "instspc tests"
from `instspc-*.test' to `*.instspc'.
* tests/instspc-data.test: Update heading comments. Execute
driver script `instspc-tests.sh', instead of sourcing it.
Give more informative messages in case of internal errors.
* tests/Makefile.am (TESTS_EXTENSIONS): Define explicitly (to
`.test' and `.instspc').
(TESTS): Add `$(instspc_tests)'.
(generated_tests): Remove `$(instspc_tests)'.
(INSTSPC_LOG_COMPILER): Define. Calls `instspc-tests.sh', so
that the "instspc tests" will be now executed by passing that
driver script a proper parameter.
($(instspc_tests)): Remove, we don't need anymore to generate
this tests.
($(instspc_tests:.test=.log)): Remove, substituted by ...
($(instspc_tests:.instspc=.log)): ... this.
($(instspc_tests)): New dependency declaration (dummy, but
required in order to have make actually produce expected log
files from the `.instspc.log' suffix rule).
(MAINTAINERCLEANFILES, generated_tests): Don't extend with
$(instspc_tests) anymore.
Update comments.
Stefano Lattarini [Wed, 16 Feb 2011 12:46:33 +0000 (13:46 +0100)]
build: improve the definition of the list of testcases
* tests/Makefile.am (handwritten_tests): New variable.
(generated_tests): Likewise.
(TESTS): Redefine as the union of the above.
(EXTRA_DIST): Extend using $(handwritten_tests) and
$(generated_tests) rather than $(TESTS).
* tests/gen-parallel-tests: Update accordingly, and
make more robust.
Stefano Lattarini [Sun, 1 May 2011 14:50:38 +0000 (16:50 +0200)]
tests skips: shorter and clearer messages
* tests/defs: Use shorter messages when giving reasons for test
skipping; it turns out these shorter messages are also clearer.
If more info might be useful, send them to the log file only.
Stefano Lattarini [Sun, 1 May 2011 13:57:06 +0000 (15:57 +0200)]
Merge branch 'master' into testsuite-work
Stefano Lattarini [Sun, 1 May 2011 13:54:46 +0000 (15:54 +0200)]
tests: fix self check spurious failure with older bash versions
* tests/self-check-cleanup.test: Add trailing `:' to the test code
passed to $SHELL, otherwise older versions of bash (at least 2.05
and 3.2.39) fail to correctly remove the temporary directory in
the exit trap.
Stefano Lattarini [Sun, 1 May 2011 12:57:47 +0000 (14:57 +0200)]
tests: add sanity check to 'self-check-cleanup.test'
* tests/self-check-cleanup.test: Check that the "hacked" file
`defs-static' used by the test differs from the "vanilla" one
in builddir. This also offers a little more debugging output.
Stefano Lattarini [Sun, 1 May 2011 08:12:51 +0000 (10:12 +0200)]
testsuite: display reasons for skips to the console
* tests/Makefile.am (AM_TESTS_ENVIRONMENT): Open file descriptor
`9' to the original stderr; define `stderr_fileno_' to `9', and
export it.
* tests/self-check-report.test: Prevent new spurious failures by
removing from the environment any definition of `stderr_fileno_'.
Stefano Lattarini [Wed, 2 Jun 2010 22:30:03 +0000 (00:30 +0200)]
testsuite: use `skip_' for skipping of tests
* tests/defs: Use the `skip_' subroutine for test skipping. Also
give sometimes more detailed messages about the reasons for the
skipping.
* tests/canon7.test: Likewise.
* tests/color.test: Likewise.
* tests/color2.test: Likewise.
* tests/compile2.test: Likewise.
* tests/dejagnu7.test: Likewise.
* tests/depcomp6.test: Likewise.
* tests/depcomp9.test: Likewise.
* tests/depcomp10.test: Likewise.
* tests/distlinks.test: Likewise.
* tests/distlinksbrk.test: Likewise.
* tests/fn99.test: Likewise.
* tests/fn99subdir.test: Likewise.
* tests/forcemiss2.test: Likewise.
* tests/fort5.test: Likewise.
* tests/gettext3.test: Likewise.
* tests/install2.test: Likewise.
* tests/instfail-info.test: Likewise.
* tests/instfail-java.test: Likewise.
* tests/instfail-libtool.test: Likewise.
* tests/instfail.test: Likewise.
* tests/instmany-mans.test: Likewise.
* tests/instmany-python.test: Likewise.
* tests/instmany.test: Likewise.
* tests/instsh3.test: Likewise.
* tests/ltinit.test: Likewise.
* tests/makej2.test: Likewise.
* tests/mdate6.test: Likewise.
* tests/mkinst3.test: Likewise.
* tests/parallel-tests3.test: Likewise.
* tests/parallel-tests-reset-term.test: Likewise.
* tests/parallel-tests-unreadable-log.test: Likewise,
* tests/python-virtualenv.test: Likewise.
* tests/remake-gnulib-remove-header.test: Likewise.
* tests/subobj9.test: Likewise.
* tests/symlink2.test: Likewise.
* tests/tar.test: Likewise.
* tests/tar2.test: Likewise.
* tests/txinfo26.test: Likewise.
* tests/vala2.test: Likewise.
* tests/vala3.test: Likewise.
* tests/vala5.test: Likewise.
* tests/vtexi4.test: Likewise.
* tests/instdir-texi.test: Likewise.
* tests/txinfo21.test: Likewise.
SquashMe
Stefano Lattarini [Fri, 29 Apr 2011 17:45:40 +0000 (19:45 +0200)]
testsuite: more environment sanitization
* tests/defs: Move the checks verifying that the variables `me',
`parallel_tests' and `required' aren't set in the environment ...
* tests/defs-static.in: ... in here, with some optimizations to
avoid useless forks. Also, do the same checks for the variables
`original_AUTOMAKE' and `original_ACLOCAL' too.
* tests/self-check-env-sanitize.test: Update.
* tests/Makefile.am (TESTS_ENVIRONMENT): Unset also variables
`original_AUTOMAKE' and `original_ACLOCAL'.
Stefano Lattarini [Sun, 24 Apr 2011 16:09:30 +0000 (18:09 +0200)]
Merge branch 'am-tests-environment-fix-merge'
Stefano Lattarini [Sun, 24 Apr 2011 16:08:44 +0000 (18:08 +0200)]
tests: fix spurious failure (non-renamed AM_TESTS_SETUP usage)
* tests/parallel-tests-reset-term.test: Use AM_TESTS_ENVIRONMENT
instead of AM_TESTS_SETUP (which has been removed in commit
v1.11-349-g12f48fa).
Fix spurious failure introduced by merge `v1.11-781-gfeed175'.
Stefano Lattarini [Sun, 24 Apr 2011 08:27:44 +0000 (10:27 +0200)]
Merge branch 'maint'
Stefano Lattarini [Sun, 24 Apr 2011 08:26:58 +0000 (10:26 +0200)]
Merge branch 'parallel-tests-testname-vpath-fix' into maint
Stefano Lattarini [Sun, 24 Apr 2011 08:22:05 +0000 (10:22 +0200)]
tests: fix spurious failure in a test on TESTS (VPATH-related)
* check-tests-in-builddir.test: When not using the parallel-tests
option, do not check that VPATH components are not present in the
displayed test name, since the simple-tests driver do not try to
strip them.
Stefano Lattarini [Sat, 23 Apr 2011 22:09:46 +0000 (00:09 +0200)]
Merge branch 'fix-env-selfcheck'
Stefano Lattarini [Sat, 23 Apr 2011 22:08:11 +0000 (00:08 +0200)]
tests: fix spurious failure in self-check-env-sanitize.test
* tests/self-check-env-sanitize.test: Open file descriptor `5'
to stdout.
Fix spurious failure introduced by merge `v1.11-788-g3b0c8d5'.
Stefano Lattarini [Sat, 23 Apr 2011 21:51:17 +0000 (23:51 +0200)]
Merge branch 'maint'
Stefano Lattarini [Sat, 23 Apr 2011 21:37:30 +0000 (23:37 +0200)]
test: self check subroutines for skipping/failing of tests
* tests/self-check-report.test: New test.
* tests/Makefile.am (TESTS): Update.
Jim Meyering [Sun, 16 Jan 2011 14:36:07 +0000 (15:36 +0100)]
test defs: new subroutines for test skipping/failing.
* tests/defs.in (Exit): Move definition of this function earlier.
(warn_, skip_, fail_, framework_failure_): New functions, inspired
to the homonyms in gnulib's tests/init.sh.
($stderr_fileno_): New global variable, used by the new functions
above.
* tests/README: Updated.
From a suggestion by Ralf Wildenhues.
Stefano Lattarini [Sat, 23 Apr 2011 09:22:40 +0000 (11:22 +0200)]
Merge branch 'maint'
Stefano Lattarini [Sat, 23 Apr 2011 08:55:49 +0000 (10:55 +0200)]
tests: fix typo (copy & paste blunder) in heading comment
* tests/maintclean-vpath.test: Correctly refer to the sister test
as `maintclean.test', not as `maintclean-vpath.test'.
Stefano Lattarini [Sat, 23 Apr 2011 08:48:34 +0000 (10:48 +0200)]
tests: remove redundant test `mclean.test'
* tests/mclean.test: Remove, it's a weak grepping test completely
superseded by the much more complete `maintclean.test'.
* tests/Makefile.am (TESTS): Update.
Stefano Lattarini [Fri, 22 Apr 2011 22:02:23 +0000 (00:02 +0200)]
Merge branch 'maint'
Stefano Lattarini [Fri, 22 Apr 2011 21:50:03 +0000 (23:50 +0200)]
testsuite: more environment sanitization
* tests/defs.in: Sanity check: abort if any of `parallel_tests'
or `required' is in the environment.
($sed_unindent_prog): Initialize to empty, to avoid interferences
from the environment.
* tests/self-check-me-in-env.test: Renamed to ...
* tests/self-check-env-sanitize.test: ... this, and extended.
* tests/Makefile.am (TESTS): Update.
(TESTS_ENVIRONMENT): Unset variables `parallel_tests' and
`required'. Adjust comments.
Stefano Lattarini [Tue, 19 Apr 2011 20:57:39 +0000 (22:57 +0200)]
tests: in self-checks, use $SHELL, not /bin/sh
* tests/self-check-me-in-env.test: Include `./defs-static' to get
the definition of `$SHELL'. Use `$SHELL' instead of `/bin/sh' to
execute the tests.
Stefano Lattarini [Tue, 19 Apr 2011 20:52:54 +0000 (22:52 +0200)]
tests: prefer AM_TESTS_ENVIRONMENT to TESTS_ENVIRONMENT
* tests/Makefile.am (TESTS_ENVIRONMENT): Renamed to ...
(AM_TESTS_ENVIRONMENT): ... this.
projects / platform / upstream / automake.git / log