1 README for security-tests project
3 ==WHAT IS======================================================================
5 security-tests is repository for testing packages from domain Security.
7 ==WHAT FOR=====================================================================
9 The security-tests repository is designed for testing packages mentioned below
10 with binaries provided for testing them:
15 libprivilege-control-test
17 security-server-tests-client-smack
18 security-server-tests-stress
19 security-server-tests-server
20 security-server-tests-api-speed
21 security-server-tests-password
22 security-server-tests-privilege
23 security-server-tests-dbus
25 security-manager-tests
29 There are also inner-tests for testing complex security-tests framework
30 mechanisms with binary:
31 security-tests-inner-test
33 ==HOW TO RUN===================================================================
35 Each test suite may be run with options:
36 --output=<output type> --output=<output type> ...
39 test-binary --output=text --output=xml --file=output.xml
40 --only-from-xml=<xml file> Run only testcases specified in XML file
41 --regexp='regexp' Only selected tests which names match regexp run
42 --start=<test id> Start from concrete test id
43 --group=<group name> Run tests only from one group
44 --runignored Run also ignored tests
45 --list Show a list of Test IDs
46 --listgroups Show a list of Test Group names
47 --only-from-xml=<xml file> Run only testcases specified in XML file
48 XML name is taken from attribute id="part1_part2" as whole.
49 If part1 is not found (no _) then it is implicitily set according to
50 suite part1 from binary tests
51 --listingroup=<group name> Show a list of Test IDS in one group
52 --allowchildlogs Allow to print logs from child process on screen.
53 When active child process will be able to print logs on stdout and
54 stderr. Both descriptors will be closed after test.
57 They can be run also by scripts:
61 Each test can end with one of three possible statuses:
66 ==HOW TO WRITE=================================================================
68 security-tests is based on extended dpl framework providing different macros.
69 Below are categories with macros listed as below:
75 --Test group registering macro-------------------------------------------------
79 RUNNER_TEST_GROUP_INIT
80 Registers group of tests. Test are registered under this group until
81 another group registering macro is called.
83 --Test registering macros------------------------------------------------------
84 Adding/removing those macro calls will add/remove test cases they provide. To
85 change tests, change body of those macro calls. Registered tests are run within
91 Function registered by this macro will be run in the same process as
92 framework. No forking allowed unless forked process does not throw
96 Function registered by this macro will be run in child process. No
97 forking allowed unless forked process does not throw any exception.
98 test_runner_multiprocess.h
99 RUNNER_MULTIPROCESS_TEST
100 Function registered by this macro will be run in the same process as
101 framework. Forking allowed. Exception of every process will be
106 Same as RUNNER_TEST but run only with smack enabled.
108 Same as RUNNER_TEST but run only with smack disabled.
109 RUNNER_CHILD_TEST_SMACK
110 Same as RUNNER_TEST_CHILD but run only with smack enabled.
111 RUNNER_CHILD_TEST_NOSMACK
112 Same as RUNNER_TEST_CHILD but run only with smack disabled.
113 RUNNER_MULTIPROCESS_TEST_SMACK
114 Same as RUNNER_TEST_MULTIPROCESS but run only with smack enabled.
115 RUNNER_MULTIPROCESS_TEST_NOSMACK
116 Same as RUNNER_TEST_MULTIPROCESS but run only with smack disabled.
118 --Assert macros----------------------------------------------------------------
119 Used within test registering macros.
121 First failed assertion throws test failed exception. If another assertions
122 fail, information about fail conditions and backtrace is cumulated and
123 presented together with already thrown exception message.
128 Assertion with message with backtrace information appended.
129 RUNNER_ASSERT_ERRNO_MSG
130 Assertion with message, error string and backtrace information
133 Assertion with error string and backtrace information appended.
135 Fail with message and backtrace information appended.
137 Assertion with backtrace information appended.
139 Assertion with message classified as ignored.
141 --Performence macros-----------------------------------------------------------
142 Used to do the time measurement.
146 RUNNER_PERF_TEST_BEGIN
147 Start time measurement.
149 End time measurement.
151 --Message macros---------------------------------------------------------------
152 Used to print error messages during test run time.
157 Print error message using red color.
159 --Collectors-------------------------------------------------------------------
160 Collectors are classes which collect test results. Each class does it differently.
161 Collectors can be registered by --output parameter (see HOW TO RUN section) but
162 there is also another collector created to write summary.
165 test_results_collector_summary.h
167 Collector writing tests summary. Call SummaryCollector::Register() to
170 --Usage example----------------------------------------------------------------
172 #include <test_runner.h>
173 #include <tests_common.h>
174 #include <summary_collector.h>
176 #include <sys/stat.h>
180 RUNNER_TEST_GROUP_INIT(foo_module)
182 RUNNER_TEST_SMACK(bar_allways_fails)
184 RUNNER_ASSERT(false);
187 RUNNER_TEST(bar_allways_passses)
192 RUNNER_TEST(bar_file1)
194 cosnt char *file = "bar_file1";
195 int fd = TEMP_FAILURE_RETRY(open(file, O_RDONLY));
196 RUNNER_ASSERT_ERRNO_MSG(fd != -1, "Cannot open " << file << " file");
200 RUNNER_CHILD_TEST_NOSMACK(bar_file2_dropped_root)
202 RUNNER_ASSERT_ERRNO(setgid(5000) == 0);
203 RUNNER_ASSERT_ERRNO(setuid(5000) == 0);
205 cosnt char *file = "bar_file2";
206 int fd = TEMP_FAILURE_RETRY(open(file, O_RDONLY));
209 RUNNER_FAIL_MSG("file " << file << "should not be opened");
211 RUNNER_ASSERT_ERRNO_MSG(errno == EACCESS,
212 "Wrong errno on opening " << " file");
215 int main(int argc, char *argv[])
217 SummaryCollector::Register();
218 return DPL::Test::TestRunnerSingleton::Instance().ExecTestRunner(argc, argv);
221 --Notes------------------------------------------------------------------------
223 While changing body of test cases, be sure to remove functions and global
224 variables if not used by any other tests.
225 Use const variables instead of #define's.
226 Use mechanisms already provided in common library.