README for security-tests project
-The project security-tests is designed for testing other packages:
-
--libsmack
--libprivilege-control
--security-server
-
-The test runner framework used by this project is dpl/test. For more
-information about dpl see project wrt-commons.
-
-The tests designed for each project are grouped in separate folders:
-1, libprivilege-control
- Sorces in folder tests/libprivilege-control-tests
- Test cases defined in tests/libprivilege-control-tests/test.cases.cpp
-2, libsmack
- Sources in folder tests/libsmack-tests
- Test cases defined in tests/libsmack-tests/test.cases.cpp
-3, security server
- Sources in folders tests/security-server-tests
- Test cases for different aspects of package divided into several
- security_server_tests_*.cpp files inside that folder.
-
-Tests are run on the tested device by executing security-tests-all.sh or each
-test suite separately:
-- libsmack-test,
-- libprivilege-control-test,
-- security-server-tests-client-smack,
-- security-server-tests-label,
-- security-server-tests-pid-reuser,
-- security-server-tests-server,
-- security-server-tests-api-speed,
-- security-server-tests-password,
-- security-server-tests-dbus.
+==WHAT IS======================================================================
+
+security-tests is repository for testing packages from domain Security.
+
+==WHAT FOR=====================================================================
+
+The security-tests repository is designed for testing packages mentioned below
+with binaries provided for testing them:
+
+libsmack
+ libsmack-test
+libprivilege-control
+ libprivilege-control-test
+security-server
+ security-server-tests-client-smack
+ security-server-tests-stress
+ security-server-tests-server
+ security-server-tests-api-speed
+ security-server-tests-password
+ security-server-tests-privilege
+ security-server-tests-dbus
+security-manager
+ security-manager-tests
+cynara
+ cynara-test
+
+==HOW TO RUN===================================================================
Each test suite may be run with options:
--output=<output type> --output=<output type> ...
- possible output types:
- --output=csv
- --output=html
- --output=tap
- --output=text
--output=xml
example:
test-binary --output=text --output=xml --file=output.xml
- --regexp='regexp' Only selected tests which names match regexp run
- --start=<test id> Start from concrete test id --group=<group name>
- Run tests only from one group
- --runignored Run also ignored tests
- --list Show a list of Test IDs
- --listgroups Show a list of Test Group names
- --only-from-xml=<xml file> Run only testcases specified in XML file
- XML name is taken from attribute id="part1_part2" as whole.
- If part1 is not found (no _) then it is implicitily set according to
- suite part1 from binary tests
- --listingroup=<group name> Show a list of Test IDS in one group
- --allowchildlogs Allow to print logs from child process on screen.
- When active child process will be able to print logs on stdout and
- stderr. Both descriptors will be closed after test.
- --help print help
-
-Apart from unit tests, the package also provides scripts designed to check:
-- CPU usage by SMACK related processes. (test-performance-check.sh, which
- internally uses the perf binary, and gives output to stdout.)
-- Time of last booting of the system. (boot-time-measure.bash)
+ --only-from-xml=<xml file> Run only testcases specified in XML file
+ --regexp='regexp' Only selected tests which names match regexp run
+ --start=<test id> Start from concrete test id
+ --group=<group name> Run tests only from one group
+ --runignored Run also ignored tests
+ --list Show a list of Test IDs
+ --listgroups Show a list of Test Group names
+ --only-from-xml=<xml file> Run only testcases specified in XML file
+ XML name is taken from attribute id="part1_part2" as whole.
+ If part1 is not found (no _) then it is implicitily set according to
+ suite part1 from binary tests
+ --listingroup=<group name> Show a list of Test IDS in one group
+ --allowchildlogs Allow to print logs from child process on screen.
+ When active child process will be able to print logs on stdout and
+ stderr. Both descriptors will be closed after test.
+ --help Print help
+
+They can be run also by scripts:
+ security-tests.sh
+ security-tests-all.sh
+
+Each test can end with one of three possible statuses:
+ FAILED
+ OK
+ IGNORED
+
+==HOW TO WRITE=================================================================
+
+security-tests is based on extended dpl framework providing different macros.
+Below are categories with macros listed as below:
+library
+ include
+ macro
+ description
+
+--Test group registering macro-------------------------------------------------
+
+dpl-test-framework
+ test_runner.h
+ RUNNER_TEST_GROUP_INIT
+ Registers group of tests. Test are registered under this group until
+ another group registering macro is called.
+
+--Test registering macros------------------------------------------------------
+Adding/removing those macro calls will add/remove test cases they provide. To
+change tests, change body of those macro calls. Registered tests are run within
+group alphabetically.
+
+dpl-test-framework
+ test_runner.h
+ RUNNER_TEST
+ Function registered by this macro will be run in the same process as
+ framework. No forking allowed unless forked process does not throw
+ any exception.
+ test_runner_child.h
+ RUNNER_CHILD_TEST
+ Function registered by this macro will be run in child process. No
+ forking allowed unless forked process does not throw any exception.
+ test_runner_multiprocess.h
+ RUNNER_MULTIPROCESS_TEST
+ Function registered by this macro will be run in the same process as
+ framework. Forking allowed. Exception of every process will be
+ registered.
+tests-common
+ tests_common.h
+ RUNNER_TEST_SMACK
+ Same as RUNNER_TEST but run only with smack enabled.
+ RUNNER_TEST_NOSMACK
+ Same as RUNNER_TEST but run only with smack disabled.
+ RUNNER_CHILD_TEST_SMACK
+ Same as RUNNER_TEST_CHILD but run only with smack enabled.
+ RUNNER_CHILD_TEST_NOSMACK
+ Same as RUNNER_TEST_CHILD but run only with smack disabled.
+ RUNNER_MULTIPROCESS_TEST_SMACK
+ Same as RUNNER_TEST_MULTIPROCESS but run only with smack enabled.
+ RUNNER_MULTIPROCESS_TEST_NOSMACK
+ Same as RUNNER_TEST_MULTIPROCESS but run only with smack disabled.
+
+--Assert macros----------------------------------------------------------------
+Used within test registering macros.
+
+dpl-test-framework
+ test_runner.h
+ RUNNER_ASSERT_MSG
+ Assertion with message with backtrace information appended.
+ RUNNER_ASSERT_ERRNO_MSG
+ Assertion with message, error string and backtrace information
+ appended.
+ RUNNER_ASSERT_ERRNO
+ Assertion with error string and backtrace information appended.
+ RUNNER_FAIL_MSG
+ Fail with message and backtrace information appended.
+ RUNNER_ASSERT
+ Assertion with backtrace information appended.
+ RUNNER_IGNORED_MSG
+ Assertion with message classified as ignored.
+
+--Performence macros-----------------------------------------------------------
+Used to do the time measurement.
+
+dpl-test-framework
+ test_runner.h
+ RUNNER_PERF_TEST_BEGIN
+ Start time measurement.
+ RUNNER_PERF_TEST_END
+ End time measurement.
+
+--Collectors-------------------------------------------------------------------
+Collectors are classes which collect test results. Each class does it differently.
+Collectors can be registered by --output parameter (see HOW TO RUN section) but
+there is also another collector created to write summary.
+
+tests-common
+ summary_collector.h
+ SummaryCollector
+ Collector writing tests summary. Call SummaryCollector::Register() to
+ register it
+
+--Usage example----------------------------------------------------------------
+
+#include <test_runner.h>
+#include <tests_common.h>
+#include <summary_collector.h>
+
+#include <sys/stat.h>
+#include <unistd.h>
+#include <fcntl.h>
+
+RUNNER_TEST_GROUP_INIT(foo_module)
+
+RUNNER_TEST_SMACK(bar_allways_fails)
+{
+ RUNNER_ASSERT(false);
+}
+
+RUNNER_TEST(bar_allways_passses)
+{
+ RUNNER_ASSERT(true);
+}
+
+RUNNER_TEST(bar_file1)
+{
+ cosnt char *file = "bar_file1";
+ int fd = TEMP_FAILURE_RETRY(open(file, O_RDONLY));
+ RUNNER_ASSERT_ERRNO_MSG(fd != -1, "Cannot open " << file << " file");
+ close(fd);
+}
+
+RUNNER_CHILD_TEST_NOSMACK(bar_file2_dropped_root)
+{
+ RUNNER_ASSERT_ERRNO(setgid(5000) == 0);
+ RUNNER_ASSERT_ERRNO(setuid(5000) == 0);
+
+ cosnt char *file = "bar_file2";
+ int fd = TEMP_FAILURE_RETRY(open(file, O_RDONLY));
+ if(fd != -1) {
+ close(fd);
+ RUNNER_FAIL_MSG("file " << file << "should not be opened");
+ }
+ RUNNER_ASSERT_ERRNO_MSG(errno == EACCESS,
+ "Wrong errno on opening " << " file");
+}
+
+int main(int argc, char *argv[])
+{
+ SummaryCollector::Register();
+ return DPL::Test::TestRunnerSingleton::Instance().ExecTestRunner(argc, argv);
+}
+
+--Notes------------------------------------------------------------------------
+
+ While changing body of test cases, be sure to remove functions and global
+variables if not used by any other tests.
+ Use const variables instead of #define's.
+ Use mechanisms already provided in common library.
projects / platform / core / test / security-tests.git / commitdiff