1 # Distributed under the OSI-approved BSD 3-Clause License. See accompanying
2 # file Copyright.txt or https://cmake.org/licensing for details.
4 #[=======================================================================[.rst:
10 This module defines functions to help use the Google Test infrastructure. Two
11 mechanisms for adding tests are provided. :command:`gtest_add_tests` has been
12 around for some time, originally via ``find_package(GTest)``.
13 :command:`gtest_discover_tests` was introduced in CMake 3.10.
15 The (older) :command:`gtest_add_tests` scans source files to identify tests.
16 This is usually effective, with some caveats, including in cross-compiling
17 environments, and makes setting additional properties on tests more convenient.
18 However, its handling of parameterized tests is less comprehensive, and it
19 requires re-running CMake to detect changes to the list of tests.
21 The (newer) :command:`gtest_discover_tests` discovers tests by asking the
22 compiled test executable to enumerate its tests. This is more robust and
23 provides better handling of parameterized tests, and does not require CMake
24 to be re-run when tests change. However, it may not work in a cross-compiling
25 environment, and setting test properties is less convenient.
27 More details can be found in the documentation of the respective functions.
29 Both commands are intended to replace use of :command:`add_test` to register
30 tests, and will create a separate CTest test for each Google Test test case.
31 Note that this is in some cases less efficient, as common set-up and tear-down
32 logic cannot be shared by multiple test cases executing in the same instance.
33 However, it provides more fine-grained pass/fail information to CTest, which is
34 usually considered as more beneficial. By default, the CTest test name is the
35 same as the Google Test name (i.e. ``suite.testcase``); see also
36 ``TEST_PREFIX`` and ``TEST_SUFFIX``.
38 .. command:: gtest_add_tests
40 Automatically add tests with CTest by scanning source code for Google Test
43 gtest_add_tests(TARGET target
46 [WORKING_DIRECTORY dir]
53 ``gtest_add_tests`` attempts to identify tests by scanning source files.
54 Although this is generally effective, it uses only a basic regular expression
55 match, which can be defeated by atypical test declarations, and is unable to
56 fully "split" parameterized tests. Additionally, it requires that CMake be
57 re-run to discover any newly added, removed or renamed tests (by default,
58 this means that CMake is re-run when any test source file is changed, but see
59 ``SKIP_DEPENDENCY``). However, it has the advantage of declaring tests at
60 CMake time, which somewhat simplifies setting additional properties on tests,
61 and always works in a cross-compiling environment.
66 Specifies the Google Test executable, which must be a known CMake
67 executable target. CMake will substitute the location of the built
68 executable when running the test.
71 When provided, only the listed files will be scanned for test cases. If
72 this option is not given, the :prop_tgt:`SOURCES` property of the
73 specified ``target`` will be used to obtain the list of sources.
75 ``EXTRA_ARGS arg1...``
76 Any extra arguments to pass on the command line to each test case.
78 ``WORKING_DIRECTORY dir``
79 Specifies the directory in which to run the discovered test cases. If this
80 option is not provided, the current binary directory is used.
82 ``TEST_PREFIX prefix``
83 Specifies a ``prefix`` to be prepended to the name of each discovered test
84 case. This can be useful when the same source files are being used in
85 multiple calls to ``gtest_add_test()`` but with different ``EXTRA_ARGS``.
87 ``TEST_SUFFIX suffix``
88 Similar to ``TEST_PREFIX`` except the ``suffix`` is appended to the name of
89 every discovered test case. Both ``TEST_PREFIX`` and ``TEST_SUFFIX`` may
93 Normally, the function creates a dependency which will cause CMake to be
94 re-run if any of the sources being scanned are changed. This is to ensure
95 that the list of discovered tests is updated. If this behavior is not
96 desired (as may be the case while actually writing the test cases), this
97 option can be used to prevent the dependency from being added.
100 The variable named by ``outVar`` will be populated in the calling scope
101 with the list of discovered test cases. This allows the caller to do
102 things like manipulate test properties of the discovered tests.
106 .. code-block:: cmake
109 add_executable(FooTest FooUnitTest.cxx)
110 gtest_add_tests(TARGET FooTest
112 TEST_LIST noArgsTests
114 gtest_add_tests(TARGET FooTest
115 EXTRA_ARGS --someArg someValue
116 TEST_SUFFIX .withArgs
117 TEST_LIST withArgsTests
119 set_tests_properties(${noArgsTests} PROPERTIES TIMEOUT 10)
120 set_tests_properties(${withArgsTests} PROPERTIES TIMEOUT 20)
122 For backward compatibility, the following form is also supported::
124 gtest_add_tests(exe args files...)
127 The path to the test executable or the name of a CMake target.
129 A ;-list of extra arguments to be passed to executable. The entire
130 list must be passed as a single argument. Enclose it in quotes,
131 or pass ``""`` for no arguments.
133 A list of source files to search for tests and test fixtures.
134 Alternatively, use ``AUTO`` to specify that ``exe`` is the name
135 of a CMake executable target whose sources should be scanned.
137 .. code-block:: cmake
140 set(FooTestArgs --foo 1 --bar 2)
141 add_executable(FooTest FooUnitTest.cxx)
142 gtest_add_tests(FooTest "${FooTestArgs}" AUTO)
144 .. command:: gtest_discover_tests
146 Automatically add tests with CTest by querying the compiled test executable
147 for available tests::
149 gtest_discover_tests(target
151 [WORKING_DIRECTORY dir]
155 [NO_PRETTY_TYPES] [NO_PRETTY_VALUES]
156 [PROPERTIES name1 value1...]
158 [DISCOVERY_TIMEOUT seconds]
160 [DISCOVERY_MODE <POST_BUILD|PRE_TEST>]
163 .. versionadded:: 3.10
165 ``gtest_discover_tests()`` sets up a post-build command on the test executable
166 that generates the list of tests by parsing the output from running the test
167 with the ``--gtest_list_tests`` argument. Compared to the source parsing
168 approach of :command:`gtest_add_tests`, this ensures that the full list of
169 tests, including instantiations of parameterized tests, is obtained. Since
170 test discovery occurs at build time, it is not necessary to re-run CMake when
171 the list of tests changes.
172 However, it requires that :prop_tgt:`CROSSCOMPILING_EMULATOR` is properly set
173 in order to function in a cross-compiling environment.
175 Additionally, setting properties on tests is somewhat less convenient, since
176 the tests are not available at CMake time. Additional test properties may be
177 assigned to the set of tests as a whole using the ``PROPERTIES`` option. If
178 more fine-grained test control is needed, custom content may be provided
179 through an external CTest script using the :prop_dir:`TEST_INCLUDE_FILES`
180 directory property. The set of discovered tests is made accessible to such a
181 script via the ``<target>_TESTS`` variable.
186 Specifies the Google Test executable, which must be a known CMake
187 executable target. CMake will substitute the location of the built
188 executable when running the test.
190 ``EXTRA_ARGS arg1...``
191 Any extra arguments to pass on the command line to each test case.
193 ``WORKING_DIRECTORY dir``
194 Specifies the directory in which to run the discovered test cases. If this
195 option is not provided, the current binary directory is used.
197 ``TEST_PREFIX prefix``
198 Specifies a ``prefix`` to be prepended to the name of each discovered test
199 case. This can be useful when the same test executable is being used in
200 multiple calls to ``gtest_discover_tests()`` but with different
203 ``TEST_SUFFIX suffix``
204 Similar to ``TEST_PREFIX`` except the ``suffix`` is appended to the name of
205 every discovered test case. Both ``TEST_PREFIX`` and ``TEST_SUFFIX`` may
209 .. versionadded:: 3.22
211 Filter expression to pass as a ``--gtest_filter`` argument during test
212 discovery. Note that the expression is a wildcard-based format that
213 matches against the original test names as used by gtest. For type or
214 value-parameterized tests, these names may be different to the potentially
215 pretty-printed test names that ``ctest`` uses.
218 By default, the type index of type-parameterized tests is replaced by the
219 actual type name in the CTest test name. If this behavior is undesirable
220 (e.g. because the type names are unwieldy), this option will suppress this
224 By default, the value index of value-parameterized tests is replaced by the
225 actual value in the CTest test name. If this behavior is undesirable
226 (e.g. because the value strings are unwieldy), this option will suppress
229 ``PROPERTIES name1 value1...``
230 Specifies additional properties to be set on all tests discovered by this
231 invocation of ``gtest_discover_tests()``.
234 Make the list of tests available in the variable ``var``, rather than the
235 default ``<target>_TESTS``. This can be useful when the same test
236 executable is being used in multiple calls to ``gtest_discover_tests()``.
237 Note that this variable is only available in CTest.
239 ``DISCOVERY_TIMEOUT num``
240 .. versionadded:: 3.10.3
242 Specifies how long (in seconds) CMake will wait for the test to enumerate
243 available tests. If the test takes longer than this, discovery (and your
244 build) will fail. Most test executables will enumerate their tests very
245 quickly, but under some exceptional circumstances, a test may require a
246 longer timeout. The default is 5. See also the ``TIMEOUT`` option of
247 :command:`execute_process`.
251 In CMake versions 3.10.1 and 3.10.2, this option was called ``TIMEOUT``.
252 This clashed with the ``TIMEOUT`` test property, which is one of the
253 common properties that would be set with the ``PROPERTIES`` keyword,
254 usually leading to legal but unintended behavior. The keyword was
255 changed to ``DISCOVERY_TIMEOUT`` in CMake 3.10.3 to address this
256 problem. The ambiguous behavior of the ``TIMEOUT`` keyword in 3.10.1
257 and 3.10.2 has not been preserved.
259 ``XML_OUTPUT_DIR dir``
260 .. versionadded:: 3.18
262 If specified, the parameter is passed along with ``--gtest_output=xml:``
263 to test executable. The actual file name is the same as the test target,
264 including prefix and suffix. This should be used instead of
265 ``EXTRA_ARGS --gtest_output=xml`` to avoid race conditions writing the
266 XML result output when using parallel test execution.
269 .. versionadded:: 3.18
271 Provides greater control over when ``gtest_discover_tests()`` performs test
272 discovery. By default, ``POST_BUILD`` sets up a post-build command
273 to perform test discovery at build time. In certain scenarios, like
274 cross-compiling, this ``POST_BUILD`` behavior is not desirable.
275 By contrast, ``PRE_TEST`` delays test discovery until just prior to test
276 execution. This way test discovery occurs in the target environment
277 where the test has a better chance at finding appropriate runtime
280 ``DISCOVERY_MODE`` defaults to the value of the
281 ``CMAKE_GTEST_DISCOVER_TESTS_DISCOVERY_MODE`` variable if it is not
282 passed when calling ``gtest_discover_tests()``. This provides a mechanism
283 for globally selecting a preferred test discovery behavior without having
284 to modify each call site.
286 #]=======================================================================]
288 # Save project's policies
290 cmake_policy(SET CMP0057 NEW) # if IN_LIST
292 #------------------------------------------------------------------------------
293 function(gtest_add_tests)
296 message(FATAL_ERROR "No arguments supplied to gtest_add_tests()")
313 set(allKeywords ${options} ${oneValueArgs} ${multiValueArgs})
316 if("${ARGV0}" IN_LIST allKeywords)
317 cmake_parse_arguments(ARGS "${options}" "${oneValueArgs}" "${multiValueArgs}" ${ARGN})
318 set(autoAddSources YES)
320 # Non-keyword syntax, convert to keyword form
322 message(FATAL_ERROR "gtest_add_tests() without keyword options requires at least 3 arguments")
324 set(ARGS_TARGET "${ARGV0}")
325 set(ARGS_EXTRA_ARGS "${ARGV1}")
326 if(NOT "${ARGV2}" STREQUAL "AUTO")
327 set(ARGS_SOURCES "${ARGV}")
328 list(REMOVE_AT ARGS_SOURCES 0 1)
332 # The non-keyword syntax allows the first argument to be an arbitrary
333 # executable rather than a target if source files are also provided. In all
334 # other cases, both forms require a target.
335 if(NOT TARGET "${ARGS_TARGET}" AND NOT ARGS_SOURCES)
336 message(FATAL_ERROR "${ARGS_TARGET} does not define an existing CMake target")
338 if(NOT ARGS_WORKING_DIRECTORY)
341 set(workDir WORKING_DIRECTORY "${ARGS_WORKING_DIRECTORY}")
345 get_property(ARGS_SOURCES TARGET ${ARGS_TARGET} PROPERTY SOURCES)
350 set(gtest_case_name_regex ".*\\( *([A-Za-z_0-9]+) *, *([A-Za-z_0-9]+) *\\).*")
351 set(gtest_test_type_regex "(TYPED_TEST|TEST)_?[FP]?")
353 foreach(source IN LISTS ARGS_SOURCES)
354 if(NOT ARGS_SKIP_DEPENDENCY)
355 set_property(DIRECTORY APPEND PROPERTY CMAKE_CONFIGURE_DEPENDS ${source})
357 file(READ "${source}" contents)
358 string(REGEX MATCHALL "${gtest_test_type_regex} *\\(([A-Za-z_0-9 ,]+)\\)" found_tests "${contents}")
359 foreach(hit ${found_tests})
360 string(REGEX MATCH "${gtest_test_type_regex}" test_type ${hit})
362 # Parameterized tests have a different signature for the filter
363 if("x${test_type}" STREQUAL "xTEST_P")
364 string(REGEX REPLACE ${gtest_case_name_regex} "*/\\1.\\2/*" gtest_test_name ${hit})
365 elseif("x${test_type}" STREQUAL "xTYPED_TEST_P")
366 string(REGEX REPLACE ${gtest_case_name_regex} "*/\\1/*.\\2" gtest_test_name ${hit})
367 elseif("x${test_type}" STREQUAL "xTEST_F" OR "x${test_type}" STREQUAL "xTEST")
368 string(REGEX REPLACE ${gtest_case_name_regex} "\\1.\\2" gtest_test_name ${hit})
369 elseif("x${test_type}" STREQUAL "xTYPED_TEST")
370 string(REGEX REPLACE ${gtest_case_name_regex} "\\1/*.\\2" gtest_test_name ${hit})
372 message(WARNING "Could not parse GTest ${hit} for adding to CTest.")
376 # Make sure tests disabled in GTest get disabled in CTest
377 if(gtest_test_name MATCHES "(^|\\.)DISABLED_")
378 # Add the disabled test if CMake is new enough
379 # Note that this check is to allow backwards compatibility so this
380 # module can be copied locally in projects to use with older CMake
382 if(CMAKE_VERSION VERSION_GREATER_EQUAL 3.8.20170401)
384 "(^|\\.)DISABLED_" "\\1"
385 orig_test_name "${gtest_test_name}"
388 ${ARGS_TEST_PREFIX}${orig_test_name}${ARGS_TEST_SUFFIX}
390 add_test(NAME ${ctest_test_name}
392 COMMAND ${ARGS_TARGET}
393 --gtest_also_run_disabled_tests
394 --gtest_filter=${gtest_test_name}
397 set_tests_properties(${ctest_test_name} PROPERTIES DISABLED TRUE)
398 list(APPEND testList ${ctest_test_name})
401 set(ctest_test_name ${ARGS_TEST_PREFIX}${gtest_test_name}${ARGS_TEST_SUFFIX})
402 add_test(NAME ${ctest_test_name}
404 COMMAND ${ARGS_TARGET}
405 --gtest_filter=${gtest_test_name}
408 list(APPEND testList ${ctest_test_name})
414 set(${ARGS_TEST_LIST} ${testList} PARENT_SCOPE)
419 #------------------------------------------------------------------------------
421 function(gtest_discover_tests TARGET)
422 cmake_parse_arguments(
424 "NO_PRETTY_TYPES;NO_PRETTY_VALUES"
425 "TEST_PREFIX;TEST_SUFFIX;WORKING_DIRECTORY;TEST_LIST;DISCOVERY_TIMEOUT;XML_OUTPUT_DIR;DISCOVERY_MODE"
426 "EXTRA_ARGS;PROPERTIES;TEST_FILTER"
430 if(NOT _WORKING_DIRECTORY)
431 set(_WORKING_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}")
434 set(_TEST_LIST ${TARGET}_TESTS)
436 if(NOT _DISCOVERY_TIMEOUT)
437 set(_DISCOVERY_TIMEOUT 5)
439 if(NOT _DISCOVERY_MODE)
440 if(NOT CMAKE_GTEST_DISCOVER_TESTS_DISCOVERY_MODE)
441 set(CMAKE_GTEST_DISCOVER_TESTS_DISCOVERY_MODE "POST_BUILD")
443 set(_DISCOVERY_MODE ${CMAKE_GTEST_DISCOVER_TESTS_DISCOVERY_MODE})
449 PROPERTY CTEST_DISCOVERED_TEST_COUNTER
456 PROPERTY CTEST_DISCOVERED_TEST_COUNTER
458 math(EXPR counter "${counter} + 1")
464 PROPERTY CTEST_DISCOVERED_TEST_COUNTER
468 # Define rule to generate test list for aforementioned test executable
469 set(ctest_file_base "${CMAKE_CURRENT_BINARY_DIR}/${TARGET}[${counter}]")
470 set(ctest_include_file "${ctest_file_base}_include.cmake")
471 set(ctest_tests_file "${ctest_file_base}_tests.cmake")
472 get_property(crosscompiling_emulator
474 PROPERTY CROSSCOMPILING_EMULATOR
477 if(_DISCOVERY_MODE STREQUAL "POST_BUILD")
479 TARGET ${TARGET} POST_BUILD
480 BYPRODUCTS "${ctest_tests_file}"
481 COMMAND "${CMAKE_COMMAND}"
482 -D "TEST_TARGET=${TARGET}"
483 -D "TEST_EXECUTABLE=$<TARGET_FILE:${TARGET}>"
484 -D "TEST_EXECUTOR=${crosscompiling_emulator}"
485 -D "TEST_WORKING_DIR=${_WORKING_DIRECTORY}"
486 -D "TEST_EXTRA_ARGS=${_EXTRA_ARGS}"
487 -D "TEST_PROPERTIES=${_PROPERTIES}"
488 -D "TEST_PREFIX=${_TEST_PREFIX}"
489 -D "TEST_SUFFIX=${_TEST_SUFFIX}"
490 -D "TEST_FILTER=${_TEST_FILTER}"
491 -D "NO_PRETTY_TYPES=${_NO_PRETTY_TYPES}"
492 -D "NO_PRETTY_VALUES=${_NO_PRETTY_VALUES}"
493 -D "TEST_LIST=${_TEST_LIST}"
494 -D "CTEST_FILE=${ctest_tests_file}"
495 -D "TEST_DISCOVERY_TIMEOUT=${_DISCOVERY_TIMEOUT}"
496 -D "TEST_XML_OUTPUT_DIR=${_XML_OUTPUT_DIR}"
497 -P "${_GOOGLETEST_DISCOVER_TESTS_SCRIPT}"
501 file(WRITE "${ctest_include_file}"
502 "if(EXISTS \"${ctest_tests_file}\")\n"
503 " include(\"${ctest_tests_file}\")\n"
505 " add_test(${TARGET}_NOT_BUILT ${TARGET}_NOT_BUILT)\n"
508 elseif(_DISCOVERY_MODE STREQUAL "PRE_TEST")
510 get_property(GENERATOR_IS_MULTI_CONFIG GLOBAL
511 PROPERTY GENERATOR_IS_MULTI_CONFIG
514 if(GENERATOR_IS_MULTI_CONFIG)
515 set(ctest_tests_file "${ctest_file_base}_tests-$<CONFIG>.cmake")
518 string(CONCAT ctest_include_content
519 "if(EXISTS \"$<TARGET_FILE:${TARGET}>\")" "\n"
520 " if(NOT EXISTS \"${ctest_tests_file}\" OR" "\n"
521 " NOT \"${ctest_tests_file}\" IS_NEWER_THAN \"$<TARGET_FILE:${TARGET}>\" OR\n"
522 " NOT \"${ctest_tests_file}\" IS_NEWER_THAN \"\${CMAKE_CURRENT_LIST_FILE}\")\n"
523 " include(\"${_GOOGLETEST_DISCOVER_TESTS_SCRIPT}\")" "\n"
524 " gtest_discover_tests_impl(" "\n"
525 " TEST_EXECUTABLE" " [==[" "$<TARGET_FILE:${TARGET}>" "]==]" "\n"
526 " TEST_EXECUTOR" " [==[" "${crosscompiling_emulator}" "]==]" "\n"
527 " TEST_WORKING_DIR" " [==[" "${_WORKING_DIRECTORY}" "]==]" "\n"
528 " TEST_EXTRA_ARGS" " [==[" "${_EXTRA_ARGS}" "]==]" "\n"
529 " TEST_PROPERTIES" " [==[" "${_PROPERTIES}" "]==]" "\n"
530 " TEST_PREFIX" " [==[" "${_TEST_PREFIX}" "]==]" "\n"
531 " TEST_SUFFIX" " [==[" "${_TEST_SUFFIX}" "]==]" "\n"
532 " TEST_FILTER" " [==[" "${_TEST_FILTER}" "]==]" "\n"
533 " NO_PRETTY_TYPES" " [==[" "${_NO_PRETTY_TYPES}" "]==]" "\n"
534 " NO_PRETTY_VALUES" " [==[" "${_NO_PRETTY_VALUES}" "]==]" "\n"
535 " TEST_LIST" " [==[" "${_TEST_LIST}" "]==]" "\n"
536 " CTEST_FILE" " [==[" "${ctest_tests_file}" "]==]" "\n"
537 " TEST_DISCOVERY_TIMEOUT" " [==[" "${_DISCOVERY_TIMEOUT}" "]==]" "\n"
538 " TEST_XML_OUTPUT_DIR" " [==[" "${_XML_OUTPUT_DIR}" "]==]" "\n"
541 " include(\"${ctest_tests_file}\")" "\n"
543 " add_test(${TARGET}_NOT_BUILT ${TARGET}_NOT_BUILT)" "\n"
547 if(GENERATOR_IS_MULTI_CONFIG)
548 foreach(_config ${CMAKE_CONFIGURATION_TYPES})
549 file(GENERATE OUTPUT "${ctest_file_base}_include-${_config}.cmake" CONTENT "${ctest_include_content}" CONDITION $<CONFIG:${_config}>)
551 file(WRITE "${ctest_include_file}" "include(\"${ctest_file_base}_include-\${CTEST_CONFIGURATION_TYPE}.cmake\")")
553 file(GENERATE OUTPUT "${ctest_file_base}_include.cmake" CONTENT "${ctest_include_content}")
554 file(WRITE "${ctest_include_file}" "include(\"${ctest_file_base}_include.cmake\")")
558 message(FATAL_ERROR "Unknown DISCOVERY_MODE: ${_DISCOVERY_MODE}")
561 # Add discovered tests to directory TEST_INCLUDE_FILES
562 set_property(DIRECTORY
563 APPEND PROPERTY TEST_INCLUDE_FILES "${ctest_include_file}"
568 ###############################################################################
570 set(_GOOGLETEST_DISCOVER_TESTS_SCRIPT
571 ${CMAKE_CURRENT_LIST_DIR}/GoogleTestAddTests.cmake
574 # Restore project's policies