1 This is a collection of tests for GDB.
3 The file gdb/README contains basic instructions on how to run the
4 testsuite, while this file documents additional options and controls
5 that are available. The GDB wiki may also have some pages with ideas
12 There are two ways to run the testsuite and pass additional parameters
13 to DejaGnu. The first is to do `make check' in the main build
14 directory and specifying the makefile variable `RUNTESTFLAGS':
16 make check RUNTESTFLAGS='TRANSCRIPT=y gdb.base/a2-run.exp'
18 The second is to cd to the testsuite directory and invoke the DejaGnu
19 `runtest' command directly.
25 (The `site.exp' file contains a handful of useful variables like host
26 and target triplets, and pathnames.)
31 If not testing with a remote host (in DejaGnu's sense), you can run
32 the GDB test suite in a fully parallel mode. In this mode, each .exp
33 file runs separately and maybe simultaneously. The test suite ensures
34 that all the temporary files created by the test suite do not clash,
35 by putting them into separate directories. This mode is primarily
36 intended for use by the Makefile.
38 For GNU make, the Makefile tries to run the tests in parallel mode if
39 any -j option is given. For a non-GNU make, tests are not
42 If RUNTESTFLAGS is not empty, then by default the tests are
43 serialized. This can be overridden by either using the
44 `check-parallel' target in the Makefile, or by setting FORCE_PARALLEL
45 to any non-empty value:
47 make check-parallel RUNTESTFLAGS="--target_board=native-gdbserver"
48 make check RUNTESTFLAGS="--target_board=native-gdbserver" FORCE_PARALLEL=1
50 If you want to use runtest directly instead of using the Makefile, see
51 the description of GDB_PARALLEL below.
56 Sometimes, new testcases are added to the testsuite that are not
57 entirely deterministic, and can randomly pass or fail. We call them
58 "racy testcases", and they can be bothersome when one is comparing
59 different testsuite runs. In order to help identifying them, it is
60 possible to run the tests several times in a row and ask the testsuite
61 machinery to analyze the results. To do that, you need to specify the
62 RACY_ITER environment variable to make:
64 make check RACY_ITER=5 -j4
66 The value assigned to RACY_ITER represents the number of times you
67 wish to run the tests in sequence (in the example above, the entire
68 testsuite will be executed 5 times in a row, in parallel). It is also
69 possible to check just a specific test:
71 make check TESTS='gdb.base/default.exp' RACY_ITER=3
73 One can also decide to call the Makefile rules by hand inside the
74 gdb/testsuite directory, e.g.:
76 make check-paralell-racy -j4
78 In which case the value of the DEFAULT_RACY_ITER variable (inside
79 gdb/testsuite/Makefile.in) will be used to determine how many
80 iterations will be run.
82 After running the tests, you shall see a file name 'racy.sum' in the
83 gdb/testsuite directory. You can also inspect the generated *.log and
84 *.sum files by looking into the gdb/testsuite/racy_ouputs directory.
86 If you already have *.sum files generated from previous testsuite runs
87 and you would like to analyze them without having to run the testsuite
88 again, you can also use the 'analyze-racy-logs.py' script directly.
89 It is located in the gdb/testsuite/ directory, and it expects a list
90 of two or more *.sum files to be provided as its argument. For
93 ./gdb/testsuite/analyze-racy-logs.py testsuite-01/gdb.sum \
94 testsuite-02/gdb.sum testsuite-03/gdb.sum
96 The script will output its analysis report to the standard output.
98 Running the Performance Tests
99 *****************************
101 GDB Testsuite includes performance test cases, which are not run together
102 with other test cases, because performance test cases are slow and need
103 a quiet system. There are two ways to run the performance test cases.
104 The first is to do `make check-perf' in the main build directory:
106 make check-perf RUNTESTFLAGS="solib.exp SOLIB_COUNT=8"
108 The second is to cd to the testsuite directory and invoke the DejaGnu
109 `runtest' command directly.
113 runtest GDB_PERFTEST_MODE=both GDB_PERFTEST_TIMEOUT=4000 --directory=gdb.perf solib.exp SOLIB_COUNT=8
115 Only "compile", "run" and "both" are valid to GDB_PERFTEST_MODE. They
116 stand for "compile tests only", "run tests only", and "compile and run
117 tests" respectively. "both" is the default. GDB_PERFTEST_TIMEOUT
118 specify the timeout, which is 3000 in default. The result of
119 performance test is appended in `testsuite/perftest.log'.
124 The following parameters are DejaGNU variables that you can set to
125 affect the testsuite run globally.
129 You may find it useful to have a transcript of the commands that the
130 testsuite sends to GDB, for instance if GDB crashes during the run,
131 and you want to reconstruct the sequence of commands.
133 If the DejaGNU variable TRANSCRIPT is set (to any value), each
134 invocation of GDB during the test run will get a transcript file
135 written into the DejaGNU output directory. The file will have the
136 name transcript.<n>, where <n> is an integer. The first line of the
137 file shows the invocation command with all the options passed to it,
138 while subsequent lines are the GDB commands. A `make check' might
141 make check RUNTESTFLAGS=TRANSCRIPT=y
143 The transcript may not be complete, as for instance tests of command
144 completion may show only partial command lines.
148 By default, the testsuite exercises the GDB in the build directory,
149 but you can set GDB to be a pathname to a different version. For
152 make check RUNTESTFLAGS=GDB=/usr/bin/gdb
154 runs the testsuite on the GDB in /usr/bin.
158 You can set GDBSERVER to be a particular GDBserver of interest, so for
161 make check RUNTESTFLAGS="GDB=/usr/bin/gdb GDBSERVER=/usr/bin/gdbserver"
163 checks both the installed GDB and GDBserver.
167 Command line options passed to all GDB invocations.
169 The default is "-nw -nx".
171 `-nw' disables any of the windowed interfaces.
172 `-nx' disables ~/.gdbinit, so that it doesn't interfere with
175 This is actually considered an internal variable, and you
176 won't normally want to change it. However, in some situations,
177 this may be tweaked as a last resort if the testsuite doesn't
178 have direct support for the specifics of your environment.
179 The testsuite does not override a value provided by the user.
181 As an example, when testing an installed GDB that has been
182 configured with `--with-system-gdbinit', like by default,
183 you do not want ~/.gdbinit to interfere with tests, but, you
184 may want the system .gdbinit file loaded. As there's no way to
185 ask the testsuite, or GDB, to load the system gdbinit but
186 not ~/.gdbinit, a workaround is then to remove `-nx' from
187 INTERNAL_GDBFLAGS, and point $HOME at a directory without
188 a .gdbinit. For example:
193 GDBSERVER=/usr/bin/gdbserver \
194 INTERNAL_GDBFLAGS=-nw
198 To use parallel testing mode without using the the Makefile, set
199 GDB_PARALLEL on the runtest command line to "yes". Before starting
200 the tests, you must ensure that the directories cache, outputs, and
201 temp in the test suite build directory are either empty or have been
202 deleted. cache in particular is used to share data across invocations
203 of runtest, and files there may affect the test results. The Makefile
204 automatically does these deletions.
208 Setting FORCE_PARALLEL to any non-empty value forces parallel testing
209 mode even if RUNTESTFLAGS is not empty.
211 FORCE_SEPARATE_MI_TTY
213 Setting FORCE_MI_SEPARATE_UI to 1 forces all MI testing to start GDB
214 in console mode, with MI running on a separate TTY, on a secondary UI
215 started with "new-ui".
219 For debugging parallel mode, it is handy to be able to see when a test
220 case writes to a file outside of its designated output directory.
222 If you have the inotify-tools package installed, you can set the
223 GDB_INOTIFY variable on the runtest command line. This will cause the
224 test suite to watch for parallel-unsafe file creations and report
225 them, both to stdout and in the test suite log file.
227 This setting is only meaningful in conjunction with GDB_PARALLEL.
231 This variable is used to specify which set of tests to run.
232 It is passed to make (not runtest) and its contents are a space separated
233 list of tests to run.
235 If using GNU make then the contents are wildcard-expanded using
236 GNU make's $(wildcard) function. Test paths must be fully specified,
237 relative to the "testsuite" subdirectory. This allows one to run all
238 tests in a subdirectory by passing "gdb.subdir/*.exp", or more simply
239 by using the check-gdb.subdir target in the Makefile.
241 If for some strange reason one wanted to run all tests that begin with
242 the letter "d" that is also possible: TESTS="*/d*.exp".
244 Do not write */*.exp to specify all tests (assuming all tests are only
245 nested one level deep, which is not necessarily true). This will pick up
246 .exp files in ancillary directories like "lib" and "config".
247 Instead write gdb.*/*.exp.
251 make -j10 check TESTS="gdb.server/[s-w]*.exp */x*.exp"
253 If not using GNU make then the value is passed directly to runtest.
254 If not specified, all tests are run.
258 This make (not runtest) variable is used to specify whether the
259 testsuite preloads the read1.so library into expect. Any non-empty
260 value means true. See "Race detection" below.
264 This variable can provide the hostname/address that should be used
265 when performing GDBserver-related tests. This is useful in some
266 situations, e.g., when you want to test the IPv6 connectivity of GDB
267 and GDBserver, or when using a different hostname/address is needed.
268 For example, to make GDB and GDBserver use IPv6-only connections, you
271 make check TESTS="gdb.server/*.exp" RUNTESTFLAGS='GDB_TEST_SOCKETHOST=tcp6:[::1]'
273 Note that only a hostname/address can be provided, without a port
278 This variable turns on the timestamp printing for each line of "make
279 check". Note that the timestamp will be printed on stdout output
280 only. In other words, there will be no timestamp output on either
281 gdb.sum and gdb.log files. If you would like to enable timestamp
282 printing, you can do:
288 You can provide a custom format for timestamp printing with this
289 variable. The format must be a string compatible with "strftime".
290 This variable is only useful when the TS variable is also provided.
291 If you would like to change the output format of the timestamp, you
294 make check TS=1 TS_FORMAT='[%b %H:%S]'
298 When set gdb debug is sent to the file gdb.debug in the test output
299 directory. It should be set to a comma separated list of gdb debug
301 For example, to turn on debugging for infrun and target, you can do:
303 make check GDB_DEBUG="infrun,target"
307 When set gdbserver debug is sent to the file gdbserver.debug in the test
308 output directory. Valid values are:
309 debug - turn on gdbserver debug.
310 remote - turn on gdbserver remote debug.
311 all - turn on all the above debug options.
312 For example, to turn on all gdbserver debugging, you can do:
314 make check GDBSERVER_DEBUG=all
319 The testsuite includes a mechanism that helps detect test races.
321 For example, say the program running under expect outputs "abcd", and
322 a test does something like this:
333 Which case happens to match depends on what expect manages to read
334 into its internal buffer in one go. If it manages to read three bytes
335 or more, then the first case matches. If it manages to read two
336 bytes, then the second case matches. If it manages to read only one
337 byte, then the third case matches.
339 To help detect these cases, the race detection mechanism preloads a
340 library into expect that forces the `read' system call to always
341 return at most 1 byte.
343 To enable this, either pass a non-empty value in the READ1 make
344 variable, or use the check-read1 make target instead of check.
348 make -j10 check-read1 TESTS="*/paginate-*.exp"
349 make -j10 check READ1="1"
351 Testsuite Configuration
352 ***********************
354 It is possible to adjust the behavior of the testsuite by defining
355 the global variables listed below, either in a `site.exp' file,
360 Defining this variable changes the default timeout duration used
361 during communication with GDB. More specifically, the global variable
362 used during testing is `timeout', but this variable gets reset to
363 `gdb_test_timeout' at the beginning of each testcase, which ensures
364 that any local change to `timeout' in a testcase does not affect
365 subsequent testcases.
367 This global variable comes in handy when the debugger is slower than
368 normal due to the testing environment, triggering unexpected `TIMEOUT'
369 test failures. Examples include when testing on a remote machine, or
370 against a system where communications are slow.
372 If not specifically defined, this variable gets automatically defined
373 to the same value as `timeout' during the testsuite initialization.
374 The default value of the timeout is defined in the file
375 `testsuite/config/unix.exp' (at least for Unix hosts; board files may
376 have their own values).
380 Defining this variable changes the default timeout duration when tests
381 under gdb.reverse directory are running. Process record and reverse
382 debugging is so slow that its tests have unexpected `TIMEOUT' test
383 failures. This global variable is useful to bump up the value of
384 `timeout' for gdb.reverse tests and doesn't cause any delay where
385 actual failures happen in the rest of the testsuite.
391 DejaGNU includes the concept of a "board file", which specifies
392 testing details for a particular target (which are often bare circuit
393 boards, thus the name).
395 In the GDB testsuite specifically, the board file may include a
396 number of "board settings" that test cases may check before deciding
397 whether to exercise a particular feature. For instance, a board
398 lacking any I/O devices, or perhaps simply having its I/O devices
399 not wired up, should set `noinferiorio'.
401 Here are the supported board settings:
403 gdb,cannot_call_functions
405 The board does not support inferior call, that is, invoking inferior
410 The board supports reverse execution.
412 gdb,no_hardware_watchpoints
414 The board does not support hardware watchpoints.
418 GDB is unable to intercept target file operations in remote and
419 perform them on the host.
423 The board is unable to provide I/O capability to the inferior.
427 A program will not return an exit code or result code (or the value
428 of the result is undefined, and should not be looked at).
432 The board does not support signals.
436 Skip time-consuming tests on the board with slow connection.
440 Skip tests related to floating point.
444 The board supports process record.
449 Commands to send to GDB every time a program is about to be run. The
450 first of these settings defines a single command as a string. The
451 second defines a TCL list of commands being a string each. The commands
452 are sent one by one in a sequence, first from `gdb_init_command', if any,
453 followed by individual commands from `gdb_init_command', if any, in this
458 The location of GDBserver. If GDBserver somewhere other than its
459 default location is used in test, specify the location of GDBserver in
460 this variable. The location is a file name for GDBserver, and may be
461 either absolute or relative to the testsuite subdirectory of the build
466 The location of the in-process agent (used for fast tracepoints and
467 other special tests). If the in-process agent of interest is anywhere
468 other than its default location, set this variable. The location is a
469 filename, and may be either absolute or relative to the testsuite
470 subdirectory of the build directory.
474 GDB does not support argument passing for inferior.
478 The board does not support type long long.
482 The board is running the monitor Cygmon.
486 The tests are running with a GDB stub.
490 Set to true if GDB can assume that letting the program run to end
491 reliably results in program exits being reported as such, as opposed
492 to, e.g., the program ending in an infinite loop or the board
493 crashing/resetting. If not set, this defaults to $use_gdb_stub. In
494 other words, native targets are assumed reliable by default, and
495 remote stubs assumed unreliable.
499 The predefined trace state variables the board has.
503 The target doesn't support thread names.
507 The flag required to force the compiler to produce position-independent
512 The flag required to force the linker to produce position-independent
517 The flag required to force the compiler to produce non-position-independent
522 When set gdb debug is sent to the file gdb.debug in the test output
523 directory. It should be set to a comma separated list of gdb debug
524 components. For example, to turn on debugging for infrun and target, set to
529 When set gdbserver debug is sent to the file gdbserver.debug in the test
530 output directory. Valid values are:
531 "debug" - turn on gdbserver debug.
532 "remote" - turn on gdbserver remote debug.
533 "all" - turn on all the above debug options.
535 Testsuite Organization
536 **********************
538 The testsuite is entirely contained in `gdb/testsuite'. The main
539 directory of the testsuite includes some makefiles and configury, but
540 these are minimal, and used for little besides cleaning up, since the
541 tests themselves handle the compilation of the programs that GDB will
544 The file `testsuite/lib/gdb.exp' contains common utility procs useful
545 for all GDB tests, while the directory testsuite/config contains
546 configuration-specific files, typically used for special-purpose
547 definitions of procs like `gdb_load' and `gdb_start'.
549 The tests themselves are to be found in directories named
550 'testsuite/gdb.* and subdirectories of those. The names of the test
551 files must always end with ".exp". DejaGNU collects the test files by
552 wildcarding in the test directories, so both subdirectories and
553 individual files typically get chosen and run in alphabetical order.
555 The following lists some notable types of subdirectories and what they
556 are for. Since DejaGNU finds test files no matter where they are
557 located, and since each test file sets up its own compilation and
558 execution environment, this organization is simply for convenience and
563 This is the base testsuite. The tests in it should apply to all
564 configurations of GDB (but generic native-only tests may live here).
565 The test programs should be in the subset of C that is both valid
570 Language-specific tests for any language besides C. Examples are
571 gdb.cp for C++ and gdb.rust for Rust.
575 Non-portable tests. The tests are specific to a specific
576 configuration (host or target), such as eCos.
580 Architecture-specific tests that are (usually) cross-platform.
584 Tests that exercise a specific GDB subsystem in more depth. For
585 instance, gdb.disasm exercises various disassemblers, while
586 gdb.stabs tests pathways through the stabs symbol reader.
590 GDB performance tests.
595 In many areas, the GDB tests are already quite comprehensive; you
596 should be able to copy existing tests to handle new cases. Be aware
597 that older tests may use obsolete practices but have not yet been
600 You should try to use `gdb_test' whenever possible, since it includes
601 cases to handle all the unexpected errors that might happen. However,
602 it doesn't cost anything to add new test procedures; for instance,
603 gdb.base/exprs.exp defines a `test_expr' that calls `gdb_test'
606 Only use `send_gdb' and `gdb_expect' when absolutely necessary. Even
607 if GDB has several valid responses to a command, you can use
608 `gdb_test_multiple'. Like `gdb_test', `gdb_test_multiple' recognizes
609 internal errors and unexpected prompts.
611 Do not write tests which expect a literal tab character from GDB. On
612 some operating systems (e.g. OpenBSD) the TTY layer expands tabs to
613 spaces, so by the time GDB's output reaches `expect' the tab is gone.
615 The source language programs do *not* need to be in a consistent
616 style. Since GDB is used to debug programs written in many different
617 styles, it's worth having a mix of styles in the testsuite; for
618 instance, some GDB bugs involving the display of source lines might
619 never manifest themselves if the test programs used GNU coding style
622 Some testcase results need more detailed explanation:
626 Use KFAIL for known problem of GDB itself. You must specify the GDB
627 bug report number, as in these sample tests:
629 kfail "gdb/13392" "continue to marker 2"
633 setup_kfail gdb/13392 "*-*-*"
634 kfail "continue to marker 2"
639 Short for "expected failure", this indicates a known problem with the
640 environment. This could include limitations of the operating system,
641 compiler version, and other components.
643 This example from gdb.base/attach-pie-misread.exp is a sanity check
644 for the target environment:
646 # On x86_64 it is commonly about 4MB.
647 if {$stub_size > 25000000} {
648 xfail "stub size $stub_size is too large"
652 You should provide bug report number for the failing component of the
653 environment, if such bug report is available, as with this example
654 referring to a GCC problem:
656 if {[test_compiler_info {gcc-[0-3]-*}]
657 || [test_compiler_info {gcc-4-[0-5]-*}]} {
658 setup_xfail "gcc/46955" *-*-*
660 gdb_test "python print ttype.template_argument(2)" "&C::c"
662 Note that it is also acceptable, and often preferable, to avoid
663 running the test at all. This is the better option if the limitation
664 is intrinsic to the environment, rather than a bug expected to be
665 fixed in the near future.
667 Local vs Remote vs Native
668 *************************
670 It's unfortunately easy to get confused in the testsuite about what's
671 native and what's not, what's remote and what's not. The confusion is
672 caused by the overlap in vocabulary between DejaGnu and GDB.
674 From a DejaGnu point of view:
676 - native: the host or target board is considered native if the its
677 triplet is the same as the build system's triplet,
679 - remote: the host or target board is considered remote if it's
680 running on a different machine, and thus require ssh, for example,
681 to run commands, versus simply running commands directly.
683 Note that they are not mutually exclusive, as you can have a remote
684 machine that has the same triplet as the build machine.
686 From a GDB point of view:
688 - native: when GDB uses system calls such as ptrace to interact
689 directly with processes on the same system its running on,
691 - remote: when GDB speaks the RSP (Remote Serial Protocol) with
692 another program doing the ptrace stuff.
694 Note that they are mutually exclusive. An inferior can only be either
695 debugged with the native target, or with the remote target a specific
698 That means that there are cases where the target is not remote for
699 DejaGnu, but is remote for GDB (e.g. running GDBserver on the same
702 You can also have a remote target for DejaGnu, but native for GDB
703 (e.g. building on x86 a GDB that runs on ARM and running the
704 testsuite with a remote host).
706 Therefore, care must be taken to check for the right kind of remote.
707 Use [is_remote target] to check whether the DejaGnu target board is
708 remote. When what you really want to know is whether GDB is using the
709 remote protocol, because feature X is only available when GDB debugs
710 natively, check gdb_protocol instead.