(or -i) command line argument to the test, or by setting GIT_TEST_OPTS
appropriately before running "make".
+-v::
--verbose::
This makes the test more verbose. Specifically, the
command being run and their output if any are also
numbers matching <pattern>. The number matched against is
simply the running count of the test within the file.
+-x::
+ Turn on shell tracing (i.e., `set -x`) during the tests
+ themselves. Implies `--verbose`.
+ Ignored in test scripts that set the variable 'test_untraceable'
+ to a non-empty value, unless it's run with a Bash version
+ supporting BASH_XTRACEFD, i.e. v4.1 or later.
+
+-d::
--debug::
This may help the person who is developing a new test.
It causes the command defined with test_debug to run.
failed tests so that you can inspect its contents after
the test finished.
+-i::
--immediate::
This causes the test to immediately exit upon the first
failed test. Cleanup commands requested with
in order to keep the state for inspection by the tester
to diagnose the bug.
+-l::
--long-tests::
This causes additional long-running tests to be run (where
available), for more exhaustive testing.
+-r::
+--run=<test-selector>::
+ Run only the subset of tests indicated by
+ <test-selector>. See section "Skipping Tests" below for
+ <test-selector> syntax.
+
--valgrind=<tool>::
Execute all Git binaries under valgrind tool <tool> and exit
with status 126 on errors (just like regular tests, this will
As the names depend on the tests' file names, it is safe to
run the tests with this option in parallel.
+-V::
+--verbose-log::
+ Write verbose output to the same logfile as `--tee`, but do
+ _not_ write it to stdout. Unlike `--tee --verbose`, this option
+ is safe to use when stdout is being consumed by a TAP parser
+ like `prove`. Implies `--tee` and `--verbose`.
+
--with-dashes::
By default tests are run without dashed forms of
commands (like git-commit) in the PATH (it only uses
Using this option with a RAM-based filesystem (such as tmpfs)
can massively speed up the test suite.
+--chain-lint::
+--no-chain-lint::
+ If --chain-lint is enabled, the test harness will check each
+ test to make sure that it properly "&&-chains" all commands (so
+ that a failure in the middle does not go unnoticed by the final
+ exit code of the test). This check is performed in addition to
+ running the tests themselves. You may also enable or disable
+ this feature by setting the GIT_TEST_CHAIN_LINT environment
+ variable to "1" or "0", respectively.
+
You can also set the GIT_TEST_INSTALLED environment variable to
the bindir of an existing git installation to test that installation.
You still need to have built this git sandbox, from which various
test, or t[0-9]{4} followed by ".$number" to say which
particular test to skip.
-Note that some tests in the existing test suite rely on previous
-test item, so you cannot arbitrarily disable one and expect the
-remainder of test to check what the test originally was intended
-to check.
+For an individual test suite --run could be used to specify that
+only some tests should be run or that some tests should be
+excluded from a run.
+
+The argument for --run is a list of individual test numbers or
+ranges with an optional negation prefix that define what tests in
+a test suite to include in the run. A range is two numbers
+separated with a dash and matches a range of tests with both ends
+been included. You may omit the first or the second number to
+mean "from the first test" or "up to the very last test"
+respectively.
+
+Optional prefix of '!' means that the test or a range of tests
+should be excluded from the run.
+
+If --run starts with an unprefixed number or range the initial
+set of tests to run is empty. If the first item starts with '!'
+all the tests are added to the initial set. After initial set is
+determined every test number or range is added or excluded from
+the set one by one, from left to right.
+
+Individual numbers or ranges could be separated either by a space
+or a comma.
+
+For example, to run only tests up to a specific test (21), one
+could do this:
+
+ $ sh ./t9200-git-cvsexport-commit.sh --run='1-21'
+
+or this:
+
+ $ sh ./t9200-git-cvsexport-commit.sh --run='-21'
+
+Common case is to run several setup tests (1, 2, 3) and then a
+specific test (21) that relies on that setup:
+
+ $ sh ./t9200-git-cvsexport-commit.sh --run='1 2 3 21'
+
+or:
+
+ $ sh ./t9200-git-cvsexport-commit.sh --run=1,2,3,21
+
+or:
+
+ $ sh ./t9200-git-cvsexport-commit.sh --run='-3 21'
+
+As noted above, the test set is built by going through the items
+from left to right, so this:
+
+ $ sh ./t9200-git-cvsexport-commit.sh --run='1-4 !3'
+
+will run tests 1, 2, and 4. Items that come later have higher
+precedence. It means that this:
+
+ $ sh ./t9200-git-cvsexport-commit.sh --run='!3 1-4'
+would just run tests from 1 to 4, including 3.
+
+You may use negation with ranges. The following will run all
+test in the test suite except from 7 up to 11:
+
+ $ sh ./t9200-git-cvsexport-commit.sh --run='!7-11'
+
+Some tests in a test suite rely on the previous tests performing
+certain actions, specifically some tests are designated as
+"setup" test, so you cannot _arbitrarily_ disable one test and
+expect the rest to function correctly.
+
+--run is mostly useful when you want to focus on a specific test
+and know what setup is needed for it. Or when you want to run
+everything up to a certain test.
+
+
+Running tests with special setups
+---------------------------------
+
+The whole test suite could be run to test some special features
+that cannot be easily covered by a few specific test cases. These
+could be enabled by running the test suite with correct GIT_TEST_
+environment set.
+
+GIT_TEST_GETTEXT_POISON=<non-empty?> turns all strings marked for
+translation into gibberish if non-empty (think "test -n"). Used for
+spotting those tests that need to be marked with a C_LOCALE_OUTPUT
+prerequisite when adding more strings for translation. See "Testing
+marked strings" in po/README for details.
+
+GIT_TEST_SPLIT_INDEX=<boolean> forces split-index mode on the whole
+test suite. Accept any boolean values that are accepted by git-config.
+
+GIT_TEST_FULL_IN_PACK_ARRAY=<boolean> exercises the uncommon
+pack-objects code path where there are more than 1024 packs even if
+the actual number of packs in repository is below this limit. Accept
+any boolean values that are accepted by git-config.
+
+GIT_TEST_OE_SIZE=<n> exercises the uncommon pack-objects code path
+where we do not cache object size in memory and read it from existing
+packs on demand. This normally only happens when the object size is
+over 2GB. This variable forces the code path on any object larger than
+<n> bytes.
+
+GIT_TEST_OE_DELTA_SIZE=<n> exercises the uncommon pack-objects code
+path where deltas larger than this limit require extra memory
+allocation for bookkeeping.
+
+GIT_TEST_VALIDATE_INDEX_CACHE_ENTRIES=<boolean> checks that cache-tree
+records are valid when the index is written out or after a merge. This
+is mostly to catch missing invalidation. Default is true.
+
+GIT_TEST_COMMIT_GRAPH=<boolean>, when true, forces the commit-graph to
+be written after every 'git commit' command, and overrides the
+'core.commitGraph' setting to true.
+
+GIT_TEST_FSMONITOR=$PWD/t7519/fsmonitor-all exercises the fsmonitor
+code path for utilizing a file system monitor to speed up detecting
+new or changed files.
+
+GIT_TEST_INDEX_VERSION=<n> exercises the index read/write code path
+for the index version specified. Can be set to any valid version
+(currently 2, 3, or 4).
+
+GIT_TEST_PRELOAD_INDEX=<boolean> exercises the preload-index code path
+by overriding the minimum number of cache entries required per thread.
+
+GIT_TEST_REBASE_USE_BUILTIN=<boolean>, when false, disables the
+builtin version of git-rebase. See 'rebase.useBuiltin' in
+git-config(1).
+
+GIT_TEST_INDEX_THREADS=<n> enables exercising the multi-threaded loading
+of the index for the whole test suite by bypassing the default number of
+cache entries and thread minimums. Setting this to 1 will make the
+index loading single threaded.
+
+GIT_TEST_MULTI_PACK_INDEX=<boolean>, when true, forces the multi-pack-
+index to be written after every 'git repack' command, and overrides the
+'core.multiPackIndex' setting to true.
Naming Tests
------------
-------------
The test script is written as a shell script. It should start
-with the standard "#!/bin/sh" with copyright notices, and an
+with the standard "#!/bin/sh", and an
assignment to variable 'test_description', like this:
#!/bin/sh
- #
- # Copyright (c) 2005 Junio C Hamano
- #
test_description='xxx test (option --frotz)
consistently when command line arguments --verbose (or -v),
--debug (or -d), and --immediate (or -i) is given.
-Do's, don'ts & things to keep in mind
--------------------------------------
+Do's & don'ts
+-------------
Here are a few examples of things you probably should and shouldn't do
when writing tests.
-Do:
+Here are the "do's:"
- Put all code inside test_expect_success and other assertions.
Windows, where the shell (MSYS bash) mangles absolute path names.
For details, see the commit message of 4114156ae9.
-Don't:
+ - Remember that inside the <script> part, the standard output and
+ standard error streams are discarded, and the test harness only
+ reports "ok" or "not ok" to the end user running the tests. Under
+ --verbose, they are shown to help debug the tests.
- - exit() within a <script> part.
+And here are the "don'ts:"
+
+ - Don't exit() within a <script> part.
The harness will catch this as a programming error of the test.
Use test_done instead if you need to stop the tests early (see
"Skipping tests" below).
- - use '! git cmd' when you want to make sure the git command exits
- with failure in a controlled way by calling "die()". Instead,
+ - Don't use '! git cmd' when you want to make sure the git command
+ exits with failure in a controlled way by calling "die()". Instead,
use 'test_must_fail git cmd'. This will signal a failure if git
dies in an unexpected way (e.g. segfault).
On the other hand, don't use test_must_fail for running regular
- platform commands; just use '! cmd'.
+ platform commands; just use '! cmd'. We are not in the business
+ of verifying that the world given to us sanely works.
+
+ - Don't feed the output of a git command to a pipe, as in:
+
+ git -C repo ls-files |
+ xargs -n 1 basename |
+ grep foo
+
+ which will discard git's exit code and may mask a crash. In the
+ above example, all exit codes are ignored except grep's.
+
+ Instead, write the output of that command to a temporary
+ file with ">" or assign it to a variable with "x=$(git ...)" rather
+ than pipe it.
+
+ - Don't use command substitution in a way that discards git's exit
+ code. When assigning to a variable, the exit code is not discarded,
+ e.g.:
- - use perl without spelling it as "$PERL_PATH". This is to help our
- friends on Windows where the platform Perl often adds CR before
+ x=$(git cat-file -p $sha) &&
+ ...
+
+ is OK because a crash in "git cat-file" will cause the "&&" chain
+ to fail, but:
+
+ test "refs/heads/foo" = "$(git symbolic-ref HEAD)"
+
+ is not OK and a crash in git could go undetected.
+
+ - Don't use perl without spelling it as "$PERL_PATH". This is to help
+ our friends on Windows where the platform Perl often adds CR before
the end of line, and they bundle Git with a version of Perl that
does not do so, whose path is specified with $PERL_PATH. Note that we
provide a "perl" function which uses $PERL_PATH under the hood, so
(but you do, for example, on a shebang line or in a sub script
created via "write_script").
- - use sh without spelling it as "$SHELL_PATH", when the script can
- be misinterpreted by broken platform shell (e.g. Solaris).
+ - Don't use sh without spelling it as "$SHELL_PATH", when the script
+ can be misinterpreted by broken platform shell (e.g. Solaris).
- - chdir around in tests. It is not sufficient to chdir to
+ - Don't chdir around in tests. It is not sufficient to chdir to
somewhere and then chdir back to the original location later in
the test, as any intermediate step can fail and abort the test,
causing the next test to start in an unexpected directory. Do so
inside a subshell if necessary.
- - Break the TAP output
+ - Don't save and verify the standard error of compound commands, i.e.
+ group commands, subshells, and shell functions (except test helper
+ functions like 'test_must_fail') like this:
+
+ ( cd dir && git cmd ) 2>error &&
+ test_cmp expect error
+
+ When running the test with '-x' tracing, then the trace of commands
+ executed in the compound command will be included in standard error
+ as well, quite possibly throwing off the subsequent checks examining
+ the output. Instead, save only the relevant git command's standard
+ error:
+
+ ( cd dir && git cmd 2>../error ) &&
+ test_cmp expect error
+
+ - Don't break the TAP output
The raw output from your test may be interpreted by a TAP harness. TAP
harnesses will ignore everything they don't know about, but don't step
their output.
You can glean some further possible issues from the TAP grammar
- (see http://search.cpan.org/perldoc?TAP::Parser::Grammar#TAP_Grammar)
+ (see https://metacpan.org/pod/TAP::Parser::Grammar#TAP-GRAMMAR)
but the best indication is to just run the tests with prove(1),
it'll complain if anything is amiss.
-Keep in mind:
-
- - Inside <script> part, the standard output and standard error
- streams are discarded, and the test harness only reports "ok" or
- "not ok" to the end user running the tests. Under --verbose, they
- are shown to help debugging the tests.
-
Skipping tests
--------------
argument. This is primarily meant for use during the
development of a new test script.
+ - debug <git-command>
+
+ Run a git command inside a debugger. This is primarily meant for
+ use when debugging a failing test script.
+
- test_done
Your test script must have test_done at the end. Its purpose
- test_have_prereq <prereq>
- Check if we have a prerequisite previously set with
- test_set_prereq. The most common use of this directly is to skip
- all the tests if we don't have some essential prerequisite:
+ Check if we have a prerequisite previously set with test_set_prereq.
+ The most common way to use this explicitly (as opposed to the
+ implicit use when an argument is passed to test_expect_*) is to skip
+ all the tests at the start of the test script if we don't have some
+ essential prerequisite:
if ! test_have_prereq PERL
then
test_expect_code 1 git merge "merge msg" B master
'
- - test_must_fail <git-command>
+ - test_must_fail [<options>] <git-command>
Run a git command and ensure it fails in a controlled way. Use
this instead of "! <git-command>". When git-command dies due to a
treats it as just another expected failure, which would let such a
bug go unnoticed.
- - test_might_fail <git-command>
+ Accepts the following options:
+
+ ok=<signal-name>[,<...>]:
+ Don't treat an exit caused by the given signal as error.
+ Multiple signals can be specified as a comma separated list.
+ Currently recognized signal names are: sigpipe, success.
+ (Don't use 'success', use 'test_might_fail' instead.)
+
+ - test_might_fail [<options>] <git-command>
Similar to test_must_fail, but tolerate success, too. Use this
instead of "<git-command> || :" to catch failures due to segv.
+ Accepts the same options as test_must_fail.
+
- test_cmp <expected> <actual>
Check whether the content of the <actual> file matches the
<expected> file. This behaves like "cmp" but produces more
helpful output when the test is run with "-v" option.
+ - test_cmp_rev <expected> <actual>
+
+ Check whether the <expected> rev points to the same commit as the
+ <actual> rev.
+
- test_line_count (= | -lt | -ge | ...) <length> <file>
Check whether a file has the length it is expected to.
...
'
+ - test_write_lines <lines>
+
+ Write <lines> on standard output, one line per argument.
+ Useful to prepare multi-line files in a compact form.
+
+ Example:
+
+ test_write_lines a b c d e f g >foo
+
+ Is a more compact equivalent of:
+ cat >foo <<-EOF
+ a
+ b
+ c
+ d
+ e
+ f
+ g
+ EOF
+
+
- test_pause
This command is useful for writing and debugging tests and must be
the symbolic link in the file system and a part that does; then only
the latter part need be protected by a SYMLINKS prerequisite (see below).
+ - test_oid_init
+
+ This function loads facts and useful object IDs related to the hash
+ algorithm(s) in use from the files in t/oid-info.
+
+ - test_oid_cache
+
+ This function reads per-hash algorithm information from standard
+ input (usually a heredoc) in the format described in
+ t/oid-info/README. This is useful for test-specific values, such as
+ object IDs, which must vary based on the hash algorithm.
+
+ Certain fixed values, such as hash sizes and common placeholder
+ object IDs, can be loaded with test_oid_init (described above).
+
+ - test_oid <key>
+
+ This function looks up a value for the hash algorithm in use, based
+ on the key given. The value must have been loaded using
+ test_oid_init or test_oid_cache. Providing an unknown key is an
+ error.
+
Prerequisites
-------------
Test is not run by root user, and an attempt to write to an
unwritable file is expected to fail correctly.
- - LIBPCRE
+ - PCRE
- Git was compiled with USE_LIBPCRE=YesPlease. Wrap any tests
+ Git was compiled with support for PCRE. Wrap any tests
that use git-grep --perl-regexp or git-grep -P in these.
+ - LIBPCRE1
+
+ Git was compiled with PCRE v1 support via
+ USE_LIBPCRE1=YesPlease. Wrap any PCRE using tests that for some
+ reason need v1 of the PCRE library instead of v2 in these.
+
+ - LIBPCRE2
+
+ Git was compiled with PCRE v2 support via
+ USE_LIBPCRE2=YesPlease. Wrap any PCRE using tests that for some
+ reason need v2 of the PCRE library instead of v1 in these.
+
- CASE_INSENSITIVE_FS
Test is run on a case insensitive file system.
Test is run on a filesystem which converts decomposed utf-8 (nfd)
to precomposed utf-8 (nfc).
+ - PTHREADS
+
+ Git wasn't compiled with NO_PTHREADS=YesPlease.
+
Tips for Writing Tests
----------------------
projects / platform / upstream / git.git / blobdiff