1 # Distributed under the OSI-approved BSD 3-Clause License. See accompanying
2 # file Copyright.txt or https://cmake.org/licensing for details.
4 #[=======================================================================[.rst:
10 This module defines functions to help use the Google Test infrastructure. Two
11 mechanisms for adding tests are provided. :command:`gtest_add_tests` has been
12 around for some time, originally via ``find_package(GTest)``.
13 :command:`gtest_discover_tests` was introduced in CMake 3.10.
15 The (older) :command:`gtest_add_tests` scans source files to identify tests.
16 This is usually effective, with some caveats, including in cross-compiling
17 environments, and makes setting additional properties on tests more convenient.
18 However, its handling of parameterized tests is less comprehensive, and it
19 requires re-running CMake to detect changes to the list of tests.
21 The (newer) :command:`gtest_discover_tests` discovers tests by asking the
22 compiled test executable to enumerate its tests. This is more robust and
23 provides better handling of parameterized tests, and does not require CMake
24 to be re-run when tests change. However, it may not work in a cross-compiling
25 environment, and setting test properties is less convenient.
27 More details can be found in the documentation of the respective functions.
29 Both commands are intended to replace use of :command:`add_test` to register
30 tests, and will create a separate CTest test for each Google Test test case.
31 Note that this is in some cases less efficient, as common set-up and tear-down
32 logic cannot be shared by multiple test cases executing in the same instance.
33 However, it provides more fine-grained pass/fail information to CTest, which is
34 usually considered as more beneficial. By default, the CTest test name is the
35 same as the Google Test name (i.e. ``suite.testcase``); see also
36 ``TEST_PREFIX`` and ``TEST_SUFFIX``.
38 .. command:: gtest_add_tests
40 Automatically add tests with CTest by scanning source code for Google Test
43 gtest_add_tests(TARGET target
46 [WORKING_DIRECTORY dir]
53 ``gtest_add_tests`` attempts to identify tests by scanning source files.
54 Although this is generally effective, it uses only a basic regular expression
55 match, which can be defeated by atypical test declarations, and is unable to
56 fully "split" parameterized tests. Additionally, it requires that CMake be
57 re-run to discover any newly added, removed or renamed tests (by default,
58 this means that CMake is re-run when any test source file is changed, but see
59 ``SKIP_DEPENDENCY``). However, it has the advantage of declaring tests at
60 CMake time, which somewhat simplifies setting additional properties on tests,
61 and always works in a cross-compiling environment.
66 Specifies the Google Test executable, which must be a known CMake
67 executable target. CMake will substitute the location of the built
68 executable when running the test.
71 When provided, only the listed files will be scanned for test cases. If
72 this option is not given, the :prop_tgt:`SOURCES` property of the
73 specified ``target`` will be used to obtain the list of sources.
75 ``EXTRA_ARGS arg1...``
76 Any extra arguments to pass on the command line to each test case.
78 ``WORKING_DIRECTORY dir``
79 Specifies the directory in which to run the discovered test cases. If this
80 option is not provided, the current binary directory is used.
82 ``TEST_PREFIX prefix``
83 Specifies a ``prefix`` to be prepended to the name of each discovered test
84 case. This can be useful when the same source files are being used in
85 multiple calls to ``gtest_add_test()`` but with different ``EXTRA_ARGS``.
87 ``TEST_SUFFIX suffix``
88 Similar to ``TEST_PREFIX`` except the ``suffix`` is appended to the name of
89 every discovered test case. Both ``TEST_PREFIX`` and ``TEST_SUFFIX`` may
93 Normally, the function creates a dependency which will cause CMake to be
94 re-run if any of the sources being scanned are changed. This is to ensure
95 that the list of discovered tests is updated. If this behavior is not
96 desired (as may be the case while actually writing the test cases), this
97 option can be used to prevent the dependency from being added.
100 The variable named by ``outVar`` will be populated in the calling scope
101 with the list of discovered test cases. This allows the caller to do
102 things like manipulate test properties of the discovered tests.
106 .. code-block:: cmake
109 add_executable(FooTest FooUnitTest.cxx)
110 gtest_add_tests(TARGET FooTest
112 TEST_LIST noArgsTests
114 gtest_add_tests(TARGET FooTest
115 EXTRA_ARGS --someArg someValue
116 TEST_SUFFIX .withArgs
117 TEST_LIST withArgsTests
119 set_tests_properties(${noArgsTests} PROPERTIES TIMEOUT 10)
120 set_tests_properties(${withArgsTests} PROPERTIES TIMEOUT 20)
122 For backward compatibility, the following form is also supported::
124 gtest_add_tests(exe args files...)
127 The path to the test executable or the name of a CMake target.
129 A ;-list of extra arguments to be passed to executable. The entire
130 list must be passed as a single argument. Enclose it in quotes,
131 or pass ``""`` for no arguments.
133 A list of source files to search for tests and test fixtures.
134 Alternatively, use ``AUTO`` to specify that ``exe`` is the name
135 of a CMake executable target whose sources should be scanned.
137 .. code-block:: cmake
140 set(FooTestArgs --foo 1 --bar 2)
141 add_executable(FooTest FooUnitTest.cxx)
142 gtest_add_tests(FooTest "${FooTestArgs}" AUTO)
144 .. command:: gtest_discover_tests
146 Automatically add tests with CTest by querying the compiled test executable
147 for available tests::
149 gtest_discover_tests(target
151 [WORKING_DIRECTORY dir]
155 [NO_PRETTY_TYPES] [NO_PRETTY_VALUES]
156 [PROPERTIES name1 value1...]
158 [DISCOVERY_TIMEOUT seconds]
160 [DISCOVERY_MODE <POST_BUILD|PRE_TEST>]
163 .. versionadded:: 3.10
165 ``gtest_discover_tests()`` sets up a post-build command on the test executable
166 that generates the list of tests by parsing the output from running the test
167 with the ``--gtest_list_tests`` argument. Compared to the source parsing
168 approach of :command:`gtest_add_tests`, this ensures that the full list of
169 tests, including instantiations of parameterized tests, is obtained. Since
170 test discovery occurs at build time, it is not necessary to re-run CMake when
171 the list of tests changes.
172 However, it requires that :prop_tgt:`CROSSCOMPILING_EMULATOR` is properly set
173 in order to function in a cross-compiling environment.
175 Additionally, setting properties on tests is somewhat less convenient, since
176 the tests are not available at CMake time. Additional test properties may be
177 assigned to the set of tests as a whole using the ``PROPERTIES`` option. If
178 more fine-grained test control is needed, custom content may be provided
179 through an external CTest script using the :prop_dir:`TEST_INCLUDE_FILES`
180 directory property. The set of discovered tests is made accessible to such a
181 script via the ``<target>_TESTS`` variable.
186 Specifies the Google Test executable, which must be a known CMake
187 executable target. CMake will substitute the location of the built
188 executable when running the test.
190 ``EXTRA_ARGS arg1...``
191 Any extra arguments to pass on the command line to each test case.
193 ``WORKING_DIRECTORY dir``
194 Specifies the directory in which to run the discovered test cases. If this
195 option is not provided, the current binary directory is used.
197 ``TEST_PREFIX prefix``
198 Specifies a ``prefix`` to be prepended to the name of each discovered test
199 case. This can be useful when the same test executable is being used in
200 multiple calls to ``gtest_discover_tests()`` but with different
203 ``TEST_SUFFIX suffix``
204 Similar to ``TEST_PREFIX`` except the ``suffix`` is appended to the name of
205 every discovered test case. Both ``TEST_PREFIX`` and ``TEST_SUFFIX`` may
209 .. versionadded:: 3.22
211 Filter expression to pass as a ``--gtest_filter`` argument during test
212 discovery. Note that the expression is a wildcard-based format that
213 matches against the original test names as used by gtest. For type or
214 value-parameterized tests, these names may be different to the potentially
215 pretty-printed test names that ``ctest`` uses.
218 By default, the type index of type-parameterized tests is replaced by the
219 actual type name in the CTest test name. If this behavior is undesirable
220 (e.g. because the type names are unwieldy), this option will suppress this
224 By default, the value index of value-parameterized tests is replaced by the
225 actual value in the CTest test name. If this behavior is undesirable
226 (e.g. because the value strings are unwieldy), this option will suppress
229 ``PROPERTIES name1 value1...``
230 Specifies additional properties to be set on all tests discovered by this
231 invocation of ``gtest_discover_tests()``.
234 Make the list of tests available in the variable ``var``, rather than the
235 default ``<target>_TESTS``. This can be useful when the same test
236 executable is being used in multiple calls to ``gtest_discover_tests()``.
237 Note that this variable is only available in CTest.
239 ``DISCOVERY_TIMEOUT num``
240 .. versionadded:: 3.10.3
242 Specifies how long (in seconds) CMake will wait for the test to enumerate
243 available tests. If the test takes longer than this, discovery (and your
244 build) will fail. Most test executables will enumerate their tests very
245 quickly, but under some exceptional circumstances, a test may require a
246 longer timeout. The default is 5. See also the ``TIMEOUT`` option of
247 :command:`execute_process`.
251 In CMake versions 3.10.1 and 3.10.2, this option was called ``TIMEOUT``.
252 This clashed with the ``TIMEOUT`` test property, which is one of the
253 common properties that would be set with the ``PROPERTIES`` keyword,
254 usually leading to legal but unintended behavior. The keyword was
255 changed to ``DISCOVERY_TIMEOUT`` in CMake 3.10.3 to address this
256 problem. The ambiguous behavior of the ``TIMEOUT`` keyword in 3.10.1
257 and 3.10.2 has not been preserved.
259 ``XML_OUTPUT_DIR dir``
260 .. versionadded:: 3.18
262 If specified, the parameter is passed along with ``--gtest_output=xml:``
263 to test executable. The actual file name is the same as the test target,
264 including prefix and suffix. This should be used instead of
265 ``EXTRA_ARGS --gtest_output=xml`` to avoid race conditions writing the
266 XML result output when using parallel test execution.
269 .. versionadded:: 3.18
271 Provides greater control over when ``gtest_discover_tests()`` performs test
272 discovery. By default, ``POST_BUILD`` sets up a post-build command
273 to perform test discovery at build time. In certain scenarios, like
274 cross-compiling, this ``POST_BUILD`` behavior is not desirable.
275 By contrast, ``PRE_TEST`` delays test discovery until just prior to test
276 execution. This way test discovery occurs in the target environment
277 where the test has a better chance at finding appropriate runtime
280 ``DISCOVERY_MODE`` defaults to the value of the
281 ``CMAKE_GTEST_DISCOVER_TESTS_DISCOVERY_MODE`` variable if it is not
282 passed when calling ``gtest_discover_tests()``. This provides a mechanism
283 for globally selecting a preferred test discovery behavior without having
284 to modify each call site.
286 #]=======================================================================]
288 # Save project's policies
290 cmake_policy(SET CMP0057 NEW) # if IN_LIST
292 #------------------------------------------------------------------------------
293 function(gtest_add_tests)
296 message(FATAL_ERROR "No arguments supplied to gtest_add_tests()")
313 set(allKeywords ${options} ${oneValueArgs} ${multiValueArgs})
316 if("${ARGV0}" IN_LIST allKeywords)
317 cmake_parse_arguments(ARGS "${options}" "${oneValueArgs}" "${multiValueArgs}" ${ARGN})
318 set(autoAddSources YES)
320 # Non-keyword syntax, convert to keyword form
322 message(FATAL_ERROR "gtest_add_tests() without keyword options requires at least 3 arguments")
324 set(ARGS_TARGET "${ARGV0}")
325 set(ARGS_EXTRA_ARGS "${ARGV1}")
326 if(NOT "${ARGV2}" STREQUAL "AUTO")
327 set(ARGS_SOURCES "${ARGV}")
328 list(REMOVE_AT ARGS_SOURCES 0 1)
332 # The non-keyword syntax allows the first argument to be an arbitrary
333 # executable rather than a target if source files are also provided. In all
334 # other cases, both forms require a target.
335 if(NOT TARGET "${ARGS_TARGET}" AND NOT ARGS_SOURCES)
336 message(FATAL_ERROR "${ARGS_TARGET} does not define an existing CMake target")
338 if(NOT ARGS_WORKING_DIRECTORY)
341 set(workDir WORKING_DIRECTORY "${ARGS_WORKING_DIRECTORY}")
345 get_property(ARGS_SOURCES TARGET ${ARGS_TARGET} PROPERTY SOURCES)
350 set(gtest_case_name_regex ".*\\( *([A-Za-z_0-9]+) *, *([A-Za-z_0-9]+) *\\).*")
351 set(gtest_test_type_regex "(TYPED_TEST|TEST_?[FP]?)")
353 foreach(source IN LISTS ARGS_SOURCES)
354 if(NOT ARGS_SKIP_DEPENDENCY)
355 set_property(DIRECTORY APPEND PROPERTY CMAKE_CONFIGURE_DEPENDS ${source})
357 file(READ "${source}" contents)
358 string(REGEX MATCHALL "${gtest_test_type_regex} *\\(([A-Za-z_0-9 ,]+)\\)" found_tests "${contents}")
359 foreach(hit ${found_tests})
360 string(REGEX MATCH "${gtest_test_type_regex}" test_type ${hit})
362 # Parameterized tests have a different signature for the filter
363 if("x${test_type}" STREQUAL "xTEST_P")
364 string(REGEX REPLACE ${gtest_case_name_regex} "*/\\1.\\2/*" gtest_test_name ${hit})
365 elseif("x${test_type}" STREQUAL "xTEST_F" OR "x${test_type}" STREQUAL "xTEST")
366 string(REGEX REPLACE ${gtest_case_name_regex} "\\1.\\2" gtest_test_name ${hit})
367 elseif("x${test_type}" STREQUAL "xTYPED_TEST")
368 string(REGEX REPLACE ${gtest_case_name_regex} "\\1/*.\\2" gtest_test_name ${hit})
370 message(WARNING "Could not parse GTest ${hit} for adding to CTest.")
374 # Make sure tests disabled in GTest get disabled in CTest
375 if(gtest_test_name MATCHES "(^|\\.)DISABLED_")
376 # Add the disabled test if CMake is new enough
377 # Note that this check is to allow backwards compatibility so this
378 # module can be copied locally in projects to use with older CMake
380 if(CMAKE_VERSION VERSION_GREATER_EQUAL 3.8.20170401)
382 "(^|\\.)DISABLED_" "\\1"
383 orig_test_name "${gtest_test_name}"
386 ${ARGS_TEST_PREFIX}${orig_test_name}${ARGS_TEST_SUFFIX}
388 add_test(NAME ${ctest_test_name}
390 COMMAND ${ARGS_TARGET}
391 --gtest_also_run_disabled_tests
392 --gtest_filter=${gtest_test_name}
395 set_tests_properties(${ctest_test_name} PROPERTIES DISABLED TRUE)
396 list(APPEND testList ${ctest_test_name})
399 set(ctest_test_name ${ARGS_TEST_PREFIX}${gtest_test_name}${ARGS_TEST_SUFFIX})
400 add_test(NAME ${ctest_test_name}
402 COMMAND ${ARGS_TARGET}
403 --gtest_filter=${gtest_test_name}
406 list(APPEND testList ${ctest_test_name})
412 set(${ARGS_TEST_LIST} ${testList} PARENT_SCOPE)
417 #------------------------------------------------------------------------------
419 function(gtest_discover_tests TARGET)
420 cmake_parse_arguments(
422 "NO_PRETTY_TYPES;NO_PRETTY_VALUES"
423 "TEST_PREFIX;TEST_SUFFIX;WORKING_DIRECTORY;TEST_LIST;DISCOVERY_TIMEOUT;XML_OUTPUT_DIR;DISCOVERY_MODE"
424 "EXTRA_ARGS;PROPERTIES;TEST_FILTER"
428 if(NOT _WORKING_DIRECTORY)
429 set(_WORKING_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}")
432 set(_TEST_LIST ${TARGET}_TESTS)
434 if(NOT _DISCOVERY_TIMEOUT)
435 set(_DISCOVERY_TIMEOUT 5)
437 if(NOT _DISCOVERY_MODE)
438 if(NOT CMAKE_GTEST_DISCOVER_TESTS_DISCOVERY_MODE)
439 set(CMAKE_GTEST_DISCOVER_TESTS_DISCOVERY_MODE "POST_BUILD")
441 set(_DISCOVERY_MODE ${CMAKE_GTEST_DISCOVER_TESTS_DISCOVERY_MODE})
447 PROPERTY CTEST_DISCOVERED_TEST_COUNTER
454 PROPERTY CTEST_DISCOVERED_TEST_COUNTER
456 math(EXPR counter "${counter} + 1")
462 PROPERTY CTEST_DISCOVERED_TEST_COUNTER
466 # Define rule to generate test list for aforementioned test executable
467 set(ctest_file_base "${CMAKE_CURRENT_BINARY_DIR}/${TARGET}[${counter}]")
468 set(ctest_include_file "${ctest_file_base}_include.cmake")
469 set(ctest_tests_file "${ctest_file_base}_tests.cmake")
470 get_property(crosscompiling_emulator
472 PROPERTY CROSSCOMPILING_EMULATOR
475 if(_DISCOVERY_MODE STREQUAL "POST_BUILD")
477 TARGET ${TARGET} POST_BUILD
478 BYPRODUCTS "${ctest_tests_file}"
479 COMMAND "${CMAKE_COMMAND}"
480 -D "TEST_TARGET=${TARGET}"
481 -D "TEST_EXECUTABLE=$<TARGET_FILE:${TARGET}>"
482 -D "TEST_EXECUTOR=${crosscompiling_emulator}"
483 -D "TEST_WORKING_DIR=${_WORKING_DIRECTORY}"
484 -D "TEST_EXTRA_ARGS=${_EXTRA_ARGS}"
485 -D "TEST_PROPERTIES=${_PROPERTIES}"
486 -D "TEST_PREFIX=${_TEST_PREFIX}"
487 -D "TEST_SUFFIX=${_TEST_SUFFIX}"
488 -D "TEST_FILTER=${_TEST_FILTER}"
489 -D "NO_PRETTY_TYPES=${_NO_PRETTY_TYPES}"
490 -D "NO_PRETTY_VALUES=${_NO_PRETTY_VALUES}"
491 -D "TEST_LIST=${_TEST_LIST}"
492 -D "CTEST_FILE=${ctest_tests_file}"
493 -D "TEST_DISCOVERY_TIMEOUT=${_DISCOVERY_TIMEOUT}"
494 -D "TEST_XML_OUTPUT_DIR=${_XML_OUTPUT_DIR}"
495 -P "${_GOOGLETEST_DISCOVER_TESTS_SCRIPT}"
499 file(WRITE "${ctest_include_file}"
500 "if(EXISTS \"${ctest_tests_file}\")\n"
501 " include(\"${ctest_tests_file}\")\n"
503 " add_test(${TARGET}_NOT_BUILT ${TARGET}_NOT_BUILT)\n"
506 elseif(_DISCOVERY_MODE STREQUAL "PRE_TEST")
508 get_property(GENERATOR_IS_MULTI_CONFIG GLOBAL
509 PROPERTY GENERATOR_IS_MULTI_CONFIG
512 if(GENERATOR_IS_MULTI_CONFIG)
513 set(ctest_tests_file "${ctest_file_base}_tests-$<CONFIG>.cmake")
516 string(CONCAT ctest_include_content
517 "if(EXISTS \"$<TARGET_FILE:${TARGET}>\")" "\n"
518 " if(NOT EXISTS \"${ctest_tests_file}\" OR" "\n"
519 " NOT \"${ctest_tests_file}\" IS_NEWER_THAN \"$<TARGET_FILE:${TARGET}>\" OR\n"
520 " NOT \"${ctest_tests_file}\" IS_NEWER_THAN \"\${CMAKE_CURRENT_LIST_FILE}\")\n"
521 " include(\"${_GOOGLETEST_DISCOVER_TESTS_SCRIPT}\")" "\n"
522 " gtest_discover_tests_impl(" "\n"
523 " TEST_EXECUTABLE" " [==[" "$<TARGET_FILE:${TARGET}>" "]==]" "\n"
524 " TEST_EXECUTOR" " [==[" "${crosscompiling_emulator}" "]==]" "\n"
525 " TEST_WORKING_DIR" " [==[" "${_WORKING_DIRECTORY}" "]==]" "\n"
526 " TEST_EXTRA_ARGS" " [==[" "${_EXTRA_ARGS}" "]==]" "\n"
527 " TEST_PROPERTIES" " [==[" "${_PROPERTIES}" "]==]" "\n"
528 " TEST_PREFIX" " [==[" "${_TEST_PREFIX}" "]==]" "\n"
529 " TEST_SUFFIX" " [==[" "${_TEST_SUFFIX}" "]==]" "\n"
530 " TEST_FILTER" " [==[" "${_TEST_FILTER}" "]==]" "\n"
531 " NO_PRETTY_TYPES" " [==[" "${_NO_PRETTY_TYPES}" "]==]" "\n"
532 " NO_PRETTY_VALUES" " [==[" "${_NO_PRETTY_VALUES}" "]==]" "\n"
533 " TEST_LIST" " [==[" "${_TEST_LIST}" "]==]" "\n"
534 " CTEST_FILE" " [==[" "${ctest_tests_file}" "]==]" "\n"
535 " TEST_DISCOVERY_TIMEOUT" " [==[" "${_DISCOVERY_TIMEOUT}" "]==]" "\n"
536 " TEST_XML_OUTPUT_DIR" " [==[" "${_XML_OUTPUT_DIR}" "]==]" "\n"
539 " include(\"${ctest_tests_file}\")" "\n"
541 " add_test(${TARGET}_NOT_BUILT ${TARGET}_NOT_BUILT)" "\n"
545 if(GENERATOR_IS_MULTI_CONFIG)
546 foreach(_config ${CMAKE_CONFIGURATION_TYPES})
547 file(GENERATE OUTPUT "${ctest_file_base}_include-${_config}.cmake" CONTENT "${ctest_include_content}" CONDITION $<CONFIG:${_config}>)
549 file(WRITE "${ctest_include_file}" "include(\"${ctest_file_base}_include-\${CTEST_CONFIGURATION_TYPE}.cmake\")")
551 file(GENERATE OUTPUT "${ctest_file_base}_include.cmake" CONTENT "${ctest_include_content}")
552 file(WRITE "${ctest_include_file}" "include(\"${ctest_file_base}_include.cmake\")")
556 message(FATAL_ERROR "Unknown DISCOVERY_MODE: ${_DISCOVERY_MODE}")
559 # Add discovered tests to directory TEST_INCLUDE_FILES
560 set_property(DIRECTORY
561 APPEND PROPERTY TEST_INCLUDE_FILES "${ctest_include_file}"
566 ###############################################################################
568 set(_GOOGLETEST_DISCOVER_TESTS_SCRIPT
569 ${CMAKE_CURRENT_LIST_DIR}/GoogleTestAddTests.cmake
572 # Restore project's policies