Testing: add Catch Unit test framework
authorMatthias Maennich <maennich@google.com>
Sun, 26 Jan 2020 20:29:56 +0000 (20:29 +0000)
committerMatthias Maennich <maennich@google.com>
Tue, 28 Jan 2020 17:16:56 +0000 (17:16 +0000)
commitc45d70d08d7526050df354da853957bbe11e5253
tree0ab4c25d6342609528f34ea8927ee5afa0707c32
parent48c8da31c0200b041f002d16122aef119c43efb4
Testing: add Catch Unit test framework

This patch adds the Catch [1] unit test framework in version v1.12.2 [2]
along with its integration into the existing build and test definition.

While there is version v2 available, v1 still supports C++98, hence we
can make use of it. The framework is distributed as a single header
file. And since it is less then 500k and it comes with a permissive
license, I decided to directly add the file rather than requiring
users/developers/distributors to satisfy the new dependency.

The integration is fairly simple: A new library libcatch.a provides the
`main` for the tests that run with Catch. The tests themselves require
to include the header as well and to link against said library.

As an example I migrated the test-kmi-whitelist test to Catch. The test
becomes a bit more structured and error reporting significantly
improved. E.g. see this intentional breakage:

 |  --- a/tests/test-kmi-whitelist.cc
 |  +++ b/tests/test-kmi-whitelist.cc
 |  @@ -140,5 +140,5 @@ TEST_CASE("WhitelistWithTwoSections", "[whitelists]")
 |     suppressions_type suppr
 |         = gen_suppr_spec_from_kernel_abi_whitelists(abi_whitelist_paths);
 |     REQUIRE(!suppr.empty());
 |  -  test_suppressions_are_consistent(suppr, "^test_symbol1$|^test_symbol2$");
 |  +  test_suppressions_are_consistent(suppr, "^test_symbol$|^test_symbol2$");

It leads to this test output:

 |  ---------------------------------------------------------------------------
 |  WhitelistWithTwoSections
 |  ---------------------------------------------------------------------------
 |  ../../tests/test-kmi-whitelist.cc:136
 |  ...........................................................................
 |
 |  ../../tests/test-kmi-whitelist.cc:81: FAILED:
 |    REQUIRE( left->get_symbol_name_not_regex_str() == expr )
 |  with expansion:
 |    "^test_symbol1$|^test_symbol2$"
 |    ==
 |    "^test_symbol$|^test_symbol2$"
 |
 |  ===========================================================================
 |  test cases:  6 |  5 passed | 1 failed
 |  assertions: 41 | 40 passed | 1 failed

[1] https://github.com/catchorg/Catch2
[2] https://github.com/catchorg/Catch2/releases/tag/v1.12.2

* tests/.gitignore: Add entry for .dirstamp
* tests/Makefile.am: Add libcatch test library and use it for
runtestkmiwhitelist.
* tests/lib/catch.cc: New test driver implementation.
* tests/lib/catch.hpp: Add Catch v1.12.2 header only test library.
* tests/test-kmi-whitelist.cc: Migrate to use Catch test framework.

Reviewed-by: Dodji Seketeli <dodji@seketeli.org>
Signed-off-by: Matthias Maennich <maennich@google.com>
tests/.gitignore
tests/Makefile.am
tests/lib/catch.cc [new file with mode: 0644]
tests/lib/catch.hpp [new file with mode: 0644]
tests/test-kmi-whitelist.cc