1 This is a collection of tests for GDB.
3 The file gdb/README contains basic instructions on how to run the
4 testsuite, while this file documents additional options and controls
5 that are available. The GDB wiki may also have some pages with ideas
12 There are two ways to run the testsuite and pass additional parameters
13 to DejaGnu. The first is to do `make check' in the main build
14 directory and specifying the makefile variable `RUNTESTFLAGS':
16 make check RUNTESTFLAGS='GDB=/usr/bin/gdb gdb.base/a2-run.exp'
18 The second is to cd to the testsuite directory and invoke the DejaGnu
19 `runtest' command directly.
23 runtest GDB=/usr/bin/gdb
25 (The `site.exp' file contains a handful of useful variables like host
26 and target triplets, and pathnames.)
31 If not testing with a remote host (in DejaGnu's sense), you can run
32 the GDB test suite in a fully parallel mode. In this mode, each .exp
33 file runs separately and maybe simultaneously. The test suite ensures
34 that all the temporary files created by the test suite do not clash,
35 by putting them into separate directories. This mode is primarily
36 intended for use by the Makefile.
38 For GNU make, the Makefile tries to run the tests in parallel mode if
39 any -j option is given. For a non-GNU make, tests are not
42 If RUNTESTFLAGS is not empty, then by default the tests are
43 serialized. This can be overridden by either using the
44 `check-parallel' target in the Makefile, or by setting FORCE_PARALLEL
45 to any non-empty value:
47 make check-parallel RUNTESTFLAGS="--target_board=native-gdbserver"
48 make check RUNTESTFLAGS="--target_board=native-gdbserver" FORCE_PARALLEL=1
50 If you want to use runtest directly instead of using the Makefile, see
51 the description of GDB_PARALLEL below.
56 Sometimes, new testcases are added to the testsuite that are not
57 entirely deterministic, and can randomly pass or fail. We call them
58 "racy testcases", and they can be bothersome when one is comparing
59 different testsuite runs. In order to help identifying them, it is
60 possible to run the tests several times in a row and ask the testsuite
61 machinery to analyze the results. To do that, you need to specify the
62 RACY_ITER environment variable to make:
64 make check RACY_ITER=5 -j4
66 The value assigned to RACY_ITER represents the number of times you
67 wish to run the tests in sequence (in the example above, the entire
68 testsuite will be executed 5 times in a row, in parallel). It is also
69 possible to check just a specific test:
71 make check TESTS='gdb.base/default.exp' RACY_ITER=3
73 One can also decide to call the Makefile rules by hand inside the
74 gdb/testsuite directory, e.g.:
76 make check-paralell-racy -j4
78 In which case the value of the DEFAULT_RACY_ITER variable (inside
79 gdb/testsuite/Makefile.in) will be used to determine how many
80 iterations will be run.
82 After running the tests, you shall see a file name 'racy.sum' in the
83 gdb/testsuite directory. You can also inspect the generated *.log and
84 *.sum files by looking into the gdb/testsuite/racy_ouputs directory.
86 If you already have *.sum files generated from previous testsuite runs
87 and you would like to analyze them without having to run the testsuite
88 again, you can also use the 'analyze-racy-logs.py' script directly.
89 It is located in the gdb/testsuite/ directory, and it expects a list
90 of two or more *.sum files to be provided as its argument. For
93 ./gdb/testsuite/analyze-racy-logs.py testsuite-01/gdb.sum \
94 testsuite-02/gdb.sum testsuite-03/gdb.sum
96 The script will output its analysis report to the standard output.
98 Running the Performance Tests
99 *****************************
101 GDB Testsuite includes performance test cases, which are not run together
102 with other test cases, because performance test cases are slow and need
103 a quiet system. There are two ways to run the performance test cases.
104 The first is to do `make check-perf' in the main build directory:
106 make check-perf RUNTESTFLAGS="solib.exp SOLIB_COUNT=8"
108 The second is to cd to the testsuite directory and invoke the DejaGnu
109 `runtest' command directly.
113 runtest GDB_PERFTEST_MODE=both GDB_PERFTEST_TIMEOUT=4000 --directory=gdb.perf solib.exp SOLIB_COUNT=8
115 Only "compile", "run" and "both" are valid to GDB_PERFTEST_MODE. They
116 stand for "compile tests only", "run tests only", and "compile and run
117 tests" respectively. "both" is the default. GDB_PERFTEST_TIMEOUT
118 specify the timeout, which is 3000 in default. The result of
119 performance test is appended in `testsuite/perftest.log'.
124 The following parameters are DejaGNU variables that you can set to
125 affect the testsuite run globally.
129 By default, the testsuite exercises the GDB in the build directory,
130 but you can set GDB to be a pathname to a different version. For
133 make check RUNTESTFLAGS=GDB=/usr/bin/gdb
135 runs the testsuite on the GDB in /usr/bin.
139 You can set GDBSERVER to be a particular GDBserver of interest, so for
142 make check RUNTESTFLAGS="GDB=/usr/bin/gdb GDBSERVER=/usr/bin/gdbserver"
144 checks both the installed GDB and GDBserver.
148 Command line options passed to all GDB invocations.
150 The default is "-nw -nx".
152 `-nw' disables any of the windowed interfaces.
153 `-nx' disables ~/.gdbinit, so that it doesn't interfere with
156 This is actually considered an internal variable, and you
157 won't normally want to change it. However, in some situations,
158 this may be tweaked as a last resort if the testsuite doesn't
159 have direct support for the specifics of your environment.
160 The testsuite does not override a value provided by the user.
162 As an example, when testing an installed GDB that has been
163 configured with `--with-system-gdbinit', like by default,
164 you do not want ~/.gdbinit to interfere with tests, but, you
165 may want the system .gdbinit file loaded. As there's no way to
166 ask the testsuite, or GDB, to load the system gdbinit but
167 not ~/.gdbinit, a workaround is then to remove `-nx' from
168 INTERNAL_GDBFLAGS, and point $HOME at a directory without
169 a .gdbinit. For example:
174 GDBSERVER=/usr/bin/gdbserver \
175 INTERNAL_GDBFLAGS=-nw
179 To use parallel testing mode without using the the Makefile, set
180 GDB_PARALLEL on the runtest command line to "yes". Before starting
181 the tests, you must ensure that the directories cache, outputs, and
182 temp in the test suite build directory are either empty or have been
183 deleted. cache in particular is used to share data across invocations
184 of runtest, and files there may affect the test results. The Makefile
185 automatically does these deletions.
189 Setting FORCE_PARALLEL to any non-empty value forces parallel testing
190 mode even if RUNTESTFLAGS is not empty.
192 FORCE_SEPARATE_MI_TTY
194 Setting FORCE_MI_SEPARATE_UI to 1 forces all MI testing to start GDB
195 in console mode, with MI running on a separate TTY, on a secondary UI
196 started with "new-ui".
200 For debugging parallel mode, it is handy to be able to see when a test
201 case writes to a file outside of its designated output directory.
203 If you have the inotify-tools package installed, you can set the
204 GDB_INOTIFY variable on the runtest command line. This will cause the
205 test suite to watch for parallel-unsafe file creations and report
206 them, both to stdout and in the test suite log file.
208 This setting is only meaningful in conjunction with GDB_PARALLEL.
212 This variable is used to specify which set of tests to run.
213 It is passed to make (not runtest) and its contents are a space separated
214 list of tests to run.
216 If using GNU make then the contents are wildcard-expanded using
217 GNU make's $(wildcard) function. Test paths must be fully specified,
218 relative to the "testsuite" subdirectory. This allows one to run all
219 tests in a subdirectory by passing "gdb.subdir/*.exp", or more simply
220 by using the check-gdb.subdir target in the Makefile.
222 If for some strange reason one wanted to run all tests that begin with
223 the letter "d" that is also possible: TESTS="*/d*.exp".
225 Do not write */*.exp to specify all tests (assuming all tests are only
226 nested one level deep, which is not necessarily true). This will pick up
227 .exp files in ancillary directories like "lib" and "config".
228 Instead write gdb.*/*.exp.
232 make -j10 check TESTS="gdb.server/[s-w]*.exp */x*.exp"
234 If not using GNU make then the value is passed directly to runtest.
235 If not specified, all tests are run.
239 This make (not runtest) variable is used to specify whether the
240 testsuite preloads the read1.so library into expect. Any non-empty
241 value means true. See "Race detection" below.
245 This variable can provide the hostname/address that should be used
246 when performing GDBserver-related tests. This is useful in some
247 situations, e.g., when you want to test the IPv6 connectivity of GDB
248 and GDBserver, or when using a different hostname/address is needed.
249 For example, to make GDB and GDBserver use IPv6-only connections, you
252 make check TESTS="gdb.server/*.exp" RUNTESTFLAGS='GDB_TEST_SOCKETHOST=tcp6:[::1]'
254 Note that only a hostname/address can be provided, without a port
259 This variable turns on the timestamp printing for each line of "make
260 check". Note that the timestamp will be printed on stdout output
261 only. In other words, there will be no timestamp output on either
262 gdb.sum and gdb.log files. If you would like to enable timestamp
263 printing, you can do:
269 You can provide a custom format for timestamp printing with this
270 variable. The format must be a string compatible with "strftime".
271 This variable is only useful when the TS variable is also provided.
272 If you would like to change the output format of the timestamp, you
275 make check TS=1 TS_FORMAT='[%b %H:%S]'
279 When set gdb debug is sent to the file gdb.debug in the test output
280 directory. It should be set to a comma separated list of gdb debug
282 For example, to turn on debugging for infrun and target, you can do:
284 make check GDB_DEBUG="infrun,target"
288 When set gdbserver debug is sent to the a file in the test output directory.
289 It should be set to a comma separated list of the following options:
290 debug - write gdbserver debug to gdbserver.debug.
291 remote - write gdbserver remote debug to gdbserver.debug.
292 replay - write a replay log to the file gdbserver.replay for use
294 Alternatively, it can be set to "all" to turn on all the above
295 For example, to turn on gdbserver debugging, you can do:
297 make check GDBSERVER_DEBUG="debug,replay"
302 The testsuite includes a mechanism that helps detect test races.
304 For example, say the program running under expect outputs "abcd", and
305 a test does something like this:
316 Which case happens to match depends on what expect manages to read
317 into its internal buffer in one go. If it manages to read three bytes
318 or more, then the first case matches. If it manages to read two
319 bytes, then the second case matches. If it manages to read only one
320 byte, then the third case matches.
322 To help detect these cases, the race detection mechanism preloads a
323 library into expect that forces the `read' system call to always
324 return at most 1 byte.
326 To enable this, either pass a non-empty value in the READ1 make
327 variable, or use the check-read1 make target instead of check.
331 make -j10 check-read1 TESTS="*/paginate-*.exp"
332 make -j10 check READ1="1"
334 Testsuite Configuration
335 ***********************
337 It is possible to adjust the behavior of the testsuite by defining
338 the global variables listed below, either in a `site.exp' file,
343 Defining this variable changes the default timeout duration used
344 during communication with GDB. More specifically, the global variable
345 used during testing is `timeout', but this variable gets reset to
346 `gdb_test_timeout' at the beginning of each testcase, which ensures
347 that any local change to `timeout' in a testcase does not affect
348 subsequent testcases.
350 This global variable comes in handy when the debugger is slower than
351 normal due to the testing environment, triggering unexpected `TIMEOUT'
352 test failures. Examples include when testing on a remote machine, or
353 against a system where communications are slow.
355 If not specifically defined, this variable gets automatically defined
356 to the same value as `timeout' during the testsuite initialization.
357 The default value of the timeout is defined in the file
358 `testsuite/config/unix.exp' (at least for Unix hosts; board files may
359 have their own values).
363 Defining this variable changes the default timeout duration when tests
364 under gdb.reverse directory are running. Process record and reverse
365 debugging is so slow that its tests have unexpected `TIMEOUT' test
366 failures. This global variable is useful to bump up the value of
367 `timeout' for gdb.reverse tests and doesn't cause any delay where
368 actual failures happen in the rest of the testsuite.
374 DejaGNU includes the concept of a "board file", which specifies
375 testing details for a particular target (which are often bare circuit
376 boards, thus the name).
378 In the GDB testsuite specifically, the board file may include a
379 number of "board settings" that test cases may check before deciding
380 whether to exercise a particular feature. For instance, a board
381 lacking any I/O devices, or perhaps simply having its I/O devices
382 not wired up, should set `noinferiorio'.
384 Here are the supported board settings:
386 gdb,cannot_call_functions
388 The board does not support inferior call, that is, invoking inferior
393 The board supports reverse execution.
395 gdb,no_hardware_watchpoints
397 The board does not support hardware watchpoints.
401 GDB is unable to intercept target file operations in remote and
402 perform them on the host.
406 The board is unable to provide I/O capability to the inferior.
410 A program will not return an exit code or result code (or the value
411 of the result is undefined, and should not be looked at).
415 The board does not support signals.
419 Skip time-consuming tests on the board with slow connection.
423 Skip tests related to floating point.
427 The board supports process record.
432 Commands to send to GDB every time a program is about to be run. The
433 first of these settings defines a single command as a string. The
434 second defines a TCL list of commands being a string each. The commands
435 are sent one by one in a sequence, first from `gdb_init_command', if any,
436 followed by individual commands from `gdb_init_command', if any, in this
441 The location of GDBserver. If GDBserver somewhere other than its
442 default location is used in test, specify the location of GDBserver in
443 this variable. The location is a file name for GDBserver, and may be
444 either absolute or relative to the testsuite subdirectory of the build
449 The location of the in-process agent (used for fast tracepoints and
450 other special tests). If the in-process agent of interest is anywhere
451 other than its default location, set this variable. The location is a
452 filename, and may be either absolute or relative to the testsuite
453 subdirectory of the build directory.
457 GDB does not support argument passing for inferior.
461 The board does not support type long long.
465 The board is running the monitor Cygmon.
469 The tests are running with a GDB stub.
473 Set to true if GDB can assume that letting the program run to end
474 reliably results in program exits being reported as such, as opposed
475 to, e.g., the program ending in an infinite loop or the board
476 crashing/resetting. If not set, this defaults to $use_gdb_stub. In
477 other words, native targets are assumed reliable by default, and
478 remote stubs assumed unreliable.
482 The predefined trace state variables the board has.
486 The target doesn't support thread names.
490 The flag required to force the compiler to produce position-independent
495 The flag required to force the linker to produce position-independent
500 The flag required to force the compiler to produce non-position-independent
505 When set gdb debug is sent to the file gdb.debug in the test output
506 directory. It should be set to a comma separated list of gdb debug
507 components. For example, to turn on debugging for infrun and target, set to
512 When set gdbserver debug is sent to the file gdbserver.debug in the test
513 output directory. For valid values see the entry for GDBSERVER_DEBUG.
515 Testsuite Organization
516 **********************
518 The testsuite is entirely contained in `gdb/testsuite'. The main
519 directory of the testsuite includes some makefiles and configury, but
520 these are minimal, and used for little besides cleaning up, since the
521 tests themselves handle the compilation of the programs that GDB will
524 The file `testsuite/lib/gdb.exp' contains common utility procs useful
525 for all GDB tests, while the directory testsuite/config contains
526 configuration-specific files, typically used for special-purpose
527 definitions of procs like `gdb_load' and `gdb_start'.
529 The tests themselves are to be found in directories named
530 'testsuite/gdb.* and subdirectories of those. The names of the test
531 files must always end with ".exp". DejaGNU collects the test files by
532 wildcarding in the test directories, so both subdirectories and
533 individual files typically get chosen and run in alphabetical order.
535 The following lists some notable types of subdirectories and what they
536 are for. Since DejaGNU finds test files no matter where they are
537 located, and since each test file sets up its own compilation and
538 execution environment, this organization is simply for convenience and
543 This is the base testsuite. The tests in it should apply to all
544 configurations of GDB (but generic native-only tests may live here).
545 The test programs should be in the subset of C that is both valid
550 Language-specific tests for any language besides C. Examples are
551 gdb.cp for C++ and gdb.rust for Rust.
555 Non-portable tests. The tests are specific to a specific
556 configuration (host or target), such as eCos.
560 Architecture-specific tests that are (usually) cross-platform.
564 Tests that exercise a specific GDB subsystem in more depth. For
565 instance, gdb.disasm exercises various disassemblers, while
566 gdb.stabs tests pathways through the stabs symbol reader.
570 GDB performance tests.
575 In many areas, the GDB tests are already quite comprehensive; you
576 should be able to copy existing tests to handle new cases. Be aware
577 that older tests may use obsolete practices but have not yet been
580 You should try to use `gdb_test' whenever possible, since it includes
581 cases to handle all the unexpected errors that might happen. However,
582 it doesn't cost anything to add new test procedures; for instance,
583 gdb.base/exprs.exp defines a `test_expr' that calls `gdb_test'
586 Only use `send_gdb' and `gdb_expect' when absolutely necessary. Even
587 if GDB has several valid responses to a command, you can use
588 `gdb_test_multiple'. Like `gdb_test', `gdb_test_multiple' recognizes
589 internal errors and unexpected prompts.
591 Do not write tests which expect a literal tab character from GDB. On
592 some operating systems (e.g. OpenBSD) the TTY layer expands tabs to
593 spaces, so by the time GDB's output reaches `expect' the tab is gone.
595 The source language programs do *not* need to be in a consistent
596 style. Since GDB is used to debug programs written in many different
597 styles, it's worth having a mix of styles in the testsuite; for
598 instance, some GDB bugs involving the display of source lines might
599 never manifest themselves if the test programs used GNU coding style
602 Some testcase results need more detailed explanation:
606 Use KFAIL for known problem of GDB itself. You must specify the GDB
607 bug report number, as in these sample tests:
609 kfail "gdb/13392" "continue to marker 2"
613 setup_kfail gdb/13392 "*-*-*"
614 kfail "continue to marker 2"
619 Short for "expected failure", this indicates a known problem with the
620 environment. This could include limitations of the operating system,
621 compiler version, and other components.
623 This example from gdb.base/attach-pie-misread.exp is a sanity check
624 for the target environment:
626 # On x86_64 it is commonly about 4MB.
627 if {$stub_size > 25000000} {
628 xfail "stub size $stub_size is too large"
632 You should provide bug report number for the failing component of the
633 environment, if such bug report is available, as with this example
634 referring to a GCC problem:
636 if {[test_compiler_info {gcc-[0-3]-*}]
637 || [test_compiler_info {gcc-4-[0-5]-*}]} {
638 setup_xfail "gcc/46955" *-*-*
640 gdb_test "python print ttype.template_argument(2)" "&C::c"
642 Note that it is also acceptable, and often preferable, to avoid
643 running the test at all. This is the better option if the limitation
644 is intrinsic to the environment, rather than a bug expected to be
645 fixed in the near future.
647 Local vs Remote vs Native
648 *************************
650 It's unfortunately easy to get confused in the testsuite about what's
651 native and what's not, what's remote and what's not. The confusion is
652 caused by the overlap in vocabulary between DejaGnu and GDB.
654 From a DejaGnu point of view:
656 - native: the host or target board is considered native if the its
657 triplet is the same as the build system's triplet,
659 - remote: the host or target board is considered remote if it's
660 running on a different machine, and thus require ssh, for example,
661 to run commands, versus simply running commands directly.
663 Note that they are not mutually exclusive, as you can have a remote
664 machine that has the same triplet as the build machine.
666 From a GDB point of view:
668 - native: when GDB uses system calls such as ptrace to interact
669 directly with processes on the same system its running on,
671 - remote: when GDB speaks the RSP (Remote Serial Protocol) with
672 another program doing the ptrace stuff.
674 Note that they are mutually exclusive. An inferior can only be either
675 debugged with the native target, or with the remote target a specific
678 That means that there are cases where the target is not remote for
679 DejaGnu, but is remote for GDB (e.g. running GDBserver on the same
682 You can also have a remote target for DejaGnu, but native for GDB
683 (e.g. building on x86 a GDB that runs on ARM and running the
684 testsuite with a remote host).
686 Therefore, care must be taken to check for the right kind of remote.
687 Use [is_remote target] to check whether the DejaGnu target board is
688 remote. When what you really want to know is whether GDB is using the
689 remote protocol, because feature X is only available when GDB debugs
690 natively, check gdb_protocol instead.