1 <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
3 <title>Cmockery</title>
6 <h1>Cmockery Unit Testing Framework</h1>
7 <p>Cmockery is a lightweight library that is used to author C unit tests.</p>
10 <li><a href="#Motivation">Motivation</a></li>
11 <li><a href="#Overview">Overview</a></li>
12 <li><a href="#TestExecution">Test Execution</a>
13 <li><a href="#ExceptionHandling">Exception Handling</a></li>
14 <li><a href="#FailureConditions">Failure Conditions</a></li>
15 <li><a href="#Assertions">Assertions</a></li>
17 <li><a href="#AssertMacros">Assert Macros</a></li>
19 <li><a href="#MemoryAllocation">Dynamic Memory Allocation</a></li>
20 <li><a href="#MockFunctions">Mock functions</a></li>
22 <li><a href="#MockFunctionsReturnValues">Return Values</a></li>
23 <li><a href="#MockFunctionsCheckingParameters">Checking Parameters</a></li>
25 <li><a href="#TestState">Test State</a></li>
26 <li><a href="#Example">Example</a></li>
29 <a name="Motivation"><h2>Motivation</h2></a>
30 <p>There are a variety of C unit testing frameworks available however many of
31 them are fairly complex and require the latest compiler technology. Some
32 development requires the use of old compilers which makes it difficult to
33 use some unit testing frameworks. In addition many unit testing frameworks
34 assume the code being tested is an application or module that is targeted to
35 the same platform that will ultimately execute the test. Because of this
36 assumption many frameworks require the inclusion of standard C library headers
37 in the code module being tested which may collide with the custom or
38 incomplete implementation of the C library utilized by the code under test.</p>
40 <p>Cmockery only requires a test application is linked with the standard C
41 library which minimizes conflicts with standard C library headers. Also,
42 Cmockery tries avoid the use of some of the newer features of C compilers.</p>
44 <p>This results in Cmockery being a relatively small library that can be used
45 to test a variety of exotic code. If a developer wishes to simply test an
46 application with the latest compiler then other unit testing frameworks maybe
49 <a name="Overview"><h2>Overview</h2></a>
50 <p>Cmockery tests are compiled into stand-alone executables and linked with
51 the Cmockery library, the standard C library and module being tested. Any
52 symbols external to the module being tested should be mocked - replaced with
53 functions that return values determined by the test - within the test
54 application. Even though significant differences may exist between the target
55 execution environment of a code module and the environment used to test the
56 code the unit testing is still valid since its goal is to test the logic of a
57 code modules at a functional level and not necessarily all of its interactions
58 with the target execution environment.</p>
60 <p>It may not be possible to compile a module into a test application without
61 some modification, therefore the preprocessor symbol <b>UNIT_TESTING</b> should
62 be defined when Cmockery unit test applications are compiled so code within the
63 module can be conditionally compiled for tests.</p>
65 <a name="TestExecution"><h2>Test Execution</h2></a>
66 <p>Cmockery unit test cases are functions with the signature
67 <b>void function(void **state)</b>. Cmockery test applications initialize a
68 table with test case function pointers using <b>unit_test*()</b> macros. This
69 table is then passed to the <b>run_tests()</b> macro to execute the tests.
71 <b>run_tests()</b> sets up the appropriate exception / signal handlers and
72 other data structures prior to running each test function. When a unit test
73 is complete <b>run_tests()</b> performs various checks to determine whether
74 the test succeeded.</p>
76 <h4>Using run_tests()</h4>
77 <a href="../src/example/run_tests.c">run_tests.c</a>
79 #include <stdarg.h>
80 #include <stddef.h>
81 #include <setjmp.h>
82 #include <cmockery.h>
84 // A test case that does nothing and succeeds.
85 void null_test_success(void **state) {
88 int main(int argc, char* argv[]) {
89 const UnitTest tests[] = {
90 unit_test(null_test_success),
92 return run_tests(tests);
96 <a name="ExceptionHandling"><h2>Exception Handling</h2></a>
97 <p>Before a test function is executed by <b>run_tests()</b>,
98 exception / signal handlers are overridden with a handler that simply
99 displays an error and exits a test function if an exception occurs. If an
100 exception occurs outside of a test function, for example in Cmockery itself,
101 the application aborts execution and returns an error code.</p>
103 <a name="FailureConditions"><h2>Failure Conditions</h2></a>
104 <p>If a failure occurs during a test function that's executed via
105 <b>run_tests()</b>, the test function is aborted and the application's
106 execution resumes with the next test function.
108 Test failures are ultimately signalled via the Cmockery function <b>fail()</b>.
109 The following events will result in the Cmockery library signalling a test
113 <li><a href="#Assertions">Assertions</a></li>
114 <li><a href="#ExceptionHandling">Exceptions</a></li>
115 <li><a href="#MemoryAllocation">Memory leaks</a></li>
116 <li><a href="#TestState">Mismatched setup and tear down functions</a></li>
117 <li><a href="#MockFunctionsReturnValues">Missing mock return values</a></li>
118 <li><a href="#MockFunctionsReturnValues">Unused mock return values</a></li>
119 <li><a href="#MockFunctionsCheckingParameters">Missing expected parameter values</a></li>
120 <li><a href="#MockFunctionsCheckingParameters">Unused expected parameter values</a></li>
124 <a name="Assertions"><h2>Assertions</h2></a>
125 <p>Runtime assert macros like the standard C library's <b>assert()</b> should
126 be redefined in modules being tested to use Cmockery's <b>mock_assert()</b>
127 function. Normally <b>mock_assert()</b> signals a
128 <a href="#FailureConditions">test failure</a>. If a function is called using
129 the <b>expect_assert_failure()</b> macro, any calls to <b>mock_assert()</b>
130 within the function will result in the execution of the test. If no
131 calls to <b>mock_assert()</b> occur during the function called via
132 <b>expect_assert_failure()</b> a test failure is signalled.</p>
134 <h4>Using mock_assert()</h4>
135 <a href="../src/example/assert_module.c">assert_module.c</a>
137 #include <assert.h>
139 // If unit testing is enabled override assert with mock_assert().
141 extern void mock_assert(const int result, const char* const expression,
142 const char * const file, const int line);
144 #define assert(expression) \
145 mock_assert((int)(expression), #expression, __FILE__, __LINE__);
146 #endif // UNIT_TESTING
148 void increment_value(int * const value) {
153 void decrement_value(int * const value) {
159 <a href="../src/example/assert_module_test.c">assert_module_test.c</a>
161 #include <stdarg.h>
162 #include <stddef.h>
163 #include <setjmp.h>
164 #include <cmockery.h>
166 extern void increment_value(int * const value);
168 /* This test case will fail but the assert is caught by run_tests() and the
169 * next test is executed. */
170 void increment_value_fail(void **state) {
171 increment_value(NULL);
174 // This test case succeeds since increment_value() asserts on the NULL pointer.
175 void increment_value_assert(void **state) {
176 expect_assert_failure(increment_value(NULL));
179 /* This test case fails since decrement_value() doesn't assert on a NULL
181 void decrement_value_fail(void **state) {
182 expect_assert_failure(decrement_value(NULL));
185 int main(int argc, char *argv[]) {
186 const UnitTest tests[] = {
187 unit_test(increment_value_fail),
188 unit_test(increment_value_assert),
189 unit_test(decrement_value_fail),
191 return run_tests(tests);
195 <h3><a name="AssertMacros">Assert Macros</a></h3>
197 <p>Cmockery provides an assortment of assert macros that tests applications
198 should use use in preference to the C standard library's assert macro. On an
199 assertion failure a Cmockery assert macro will write the failure to the
200 standard error stream and signal a test failure. Due to limitations of the
201 C language the general C standard library assert() and Cmockery's
202 assert_true() and assert_false() macros can only display the expression that
203 caused the assert failure. Cmockery's type specific assert macros,
204 assert_{type}_equal() and assert_{type}_not_equal(), display the data that
205 caused the assertion failure which increases data visibility aiding
206 debugging of failing test cases.</p>
208 <h4>Using assert_{type}_equal() macros</h4>
209 <a href="../src/example/assert_macro.c">assert_macro.c</a>
211 #include <string.h>
213 static const char* status_code_strings[] = {
215 "Connection dropped",
216 "Connection timed out",
219 const char* get_status_code_string(const unsigned int status_code) {
220 return status_code_strings[status_code];
223 unsigned int string_to_status_code(const char* const status_code_string) {
225 for (i = 0; i < sizeof(status_code_strings) /
226 sizeof(status_code_strings[0]); i++) {
227 if (strcmp(status_code_strings[i], status_code_string) == 0) {
234 <a href="../src/example/assert_macro_test.c">assert_macro_test.c</a>
236 #include <stdarg.h>
237 #include <stddef.h>
238 #include <setjmp.h>
239 #include <cmockery.h>
241 extern const char* get_status_code_string(const unsigned int status_code);
242 extern unsigned int string_to_status_code(
243 const char* const status_code_string);
245 /* This test will fail since the string returned by get_status_code_string(0)
246 * doesn't match "Connection timed out". */
247 void get_status_code_string_test(void **state) {
248 assert_string_equal(get_status_code_string(0), "Address not found");
249 assert_string_equal(get_status_code_string(1), "Connection timed out");
252 // This test will fail since the status code of "Connection timed out" isn't 1
253 void string_to_status_code_test(void **state) {
254 assert_int_equal(string_to_status_code("Address not found"), 0);
255 assert_int_equal(string_to_status_code("Connection timed out"), 1);
258 int main(int argc, char *argv[]) {
259 const UnitTest tests[] = {
260 unit_test(get_status_code_string_test),
261 unit_test(string_to_status_code_test),
263 return run_tests(tests);
267 <a name="MemoryAllocation"><h2>Dynamic Memory Allocation</h2></a>
269 <p>To test for memory leaks, buffer overflows and underflows a module being
270 tested by Cmockery should replace calls to <b>malloc()</b>, <b>calloc()</b> and
271 <b>free()</b> to <b>test_malloc()</b>, <b>test_calloc()</b> and
272 <b>test_free()</b> respectively. Each time a block is deallocated using
273 <b>test_free()</b> it is checked for corruption, if a corrupt block is found
274 a <a href="#FailureConditions">test failure</a> is signalled. All blocks
275 allocated using the <b>test_*()</b> allocation functions are tracked by the
276 Cmockery library. When a test completes if any allocated blocks (memory leaks)
277 remain they are reported and a test failure is signalled.</p>
278 <p>For simplicity Cmockery currently executes all tests in one process.
279 Therefore all test cases in a test application share a single address space
280 which means memory corruption from a single test case could potentially cause
281 the test application to exit prematurely.</p>
283 <h4>Using Cmockery's Allocators</h4>
284 <a href="../src/example/allocate_module.c">allocate_module.c</a>
286 #include <malloc.h>
289 extern void* _test_malloc(const size_t size, const char* file, const int line);
290 extern void* _test_calloc(const size_t number_of_elements, const size_t size,
291 const char* file, const int line);
292 extern void _test_free(void* const ptr, const char* file, const int line);
294 #define malloc(size) _test_malloc(size, __FILE__, __LINE__)
295 #define calloc(num, size) _test_calloc(num, size, __FILE__, __LINE__)
296 #define free(ptr) _test_free(ptr, __FILE__, __LINE__)
297 #endif // UNIT_TESTING
300 int * const temporary = (int*)malloc(sizeof(int));
304 void buffer_overflow() {
305 char * const memory = (char*)malloc(sizeof(int));
306 memory[sizeof(int)] = '!';
310 void buffer_underflow() {
311 char * const memory = (char*)malloc(sizeof(int));
316 <a href="../src/example/allocate_module_test.c">allocate_module_test.c</a>
318 #include <stdarg.h>
319 #include <stddef.h>
320 #include <setjmp.h>
321 #include <cmockery.h>
323 extern void leak_memory();
324 extern void buffer_overflow();
325 extern void buffer_underflow();
327 // Test case that fails as leak_memory() leaks a dynamically allocated block.
328 void leak_memory_test(void **state) {
332 // Test case that fails as buffer_overflow() corrupts an allocated block.
333 void buffer_overflow_test(void **state) {
337 // Test case that fails as buffer_underflow() corrupts an allocated block.
338 void buffer_underflow_test(void **state) {
342 int main(int argc, char* argv[]) {
343 const UnitTest tests[] = {
344 unit_test(leak_memory_test),
345 unit_test(buffer_overflow_test),
346 unit_test(buffer_underflow_test),
348 return run_tests(tests);
352 <a name="MockFunctions"><h2>Mock Functions</h2></a>
354 <p>A unit test should ideally isolate the function or module being tested
355 from any external dependencies. This can be performed using mock functions
356 that are either statically or dynamically linked with the module being tested.
357 Mock functions must be statically linked when the code being tested directly
358 references external functions. Dynamic linking is simply the process of
359 setting a function pointer in a table used by the tested module to reference
360 a mock function defined in the unit test.</p>
362 <a name="MockFunctionsReturnValues"><h3>Return Values</h3></a>
364 <p>In order to simplify the implementation of mock functions Cmockery provides
365 functionality which stores return values for mock functions in each test
366 case using <b>will_return()</b>. These values are then returned by each mock
367 function using calls to <b>mock()</b>.
369 Values passed to <b>will_return()</b> are added to a queue for each function
370 specified. Each successive call to <b>mock()</b> from a function removes a
371 return value from the queue. This makes it possible for a mock function to use
372 multiple calls to <b>mock()</b> to return output parameters in addition to a
373 return value. In addition this allows the specification of return values for
374 multiple calls to a mock function.</p>
376 <h4>Using will_return()</h4>
377 <a name="../src/example/database.h" href="database.h">database.h</a>
379 typedef struct DatabaseConnection DatabaseConnection;
381 /* Function that takes an SQL query string and sets results to an array of
382 * pointers with the result of the query. The value returned specifies the
383 * number of items in the returned array of results. The returned array of
384 * results are statically allocated and should not be deallocated using free()
386 typedef unsigned int (*QueryDatabase)(
387 DatabaseConnection* const connection, const char * const query_string,
388 void *** const results);
390 // Connection to a database.
391 struct DatabaseConnection {
394 QueryDatabase query_database;
397 // Connect to a database.
398 DatabaseConnection* connect_to_database(const char * const url,
399 const unsigned int port);
401 <a href="../src/example/customer_database.c">customer_database.c</a>
403 #include <stddef.h>
404 #include <stdio.h>
405 #include <database.h>
407 #define snprintf _snprintf
410 // Connect to the database containing customer information.
411 DatabaseConnection* connect_to_customer_database() {
412 return connect_to_database("customers.abcd.org", 321);
415 /* Find the ID of a customer by his/her name returning a value > 0 if
416 * successful, 0 otherwise. */
417 unsigned int get_customer_id_by_name(
418 DatabaseConnection * const connection,
419 const char * const customer_name) {
420 char query_string[256];
421 int number_of_results;
423 snprintf(query_string, sizeof(query_string),
424 "SELECT ID FROM CUSTOMERS WHERE NAME = %s", customer_name);
425 number_of_results = connection->query_database(connection, query_string,
427 if (number_of_results != 1) {
430 return (unsigned int)results[0];
433 <a href="../src/example/customer_database_test.c">customer_database_test.c</a>
435 #include <stdarg.h>
436 #include <stddef.h>
437 #include <setjmp.h>
438 #include <cmockery.h>
439 #include <database.h>
442 extern DatabaseConnection* connect_to_customer_database();
443 extern unsigned int get_customer_id_by_name(
444 DatabaseConnection * const connection, const char * const customer_name);
446 // Mock query database function.
447 unsigned int mock_query_database(
448 DatabaseConnection* const connection, const char * const query_string,
449 void *** const results) {
450 *results = (void**)mock();
451 return (unsigned int)mock();
454 // Mock of the connect to database function.
455 DatabaseConnection* connect_to_database(const char * const database_url,
456 const unsigned int port) {
457 return (DatabaseConnection*)mock();
460 void test_connect_to_customer_database(void **state) {
461 will_return(connect_to_database, 0x0DA7ABA53);
462 assert_true(connect_to_customer_database() ==
463 (DatabaseConnection*)0x0DA7ABA53);
466 /* This test fails as the mock function connect_to_database() will have no
467 * value to return. */
468 void fail_connect_to_customer_database(void **state) {
469 will_return(connect_to_database, 0x0DA7ABA53);
470 assert_true(connect_to_customer_database() ==
471 (DatabaseConnection*)0x0DA7ABA53);
474 void test_get_customer_id_by_name(void **state) {
475 DatabaseConnection connection = {
476 "somedatabase.somewhere.com", 12345678, mock_query_database
478 // Return a single customer ID when mock_query_database() is called.
479 int customer_ids = 543;
480 will_return(mock_query_database, &customer_ids);
481 will_return(mock_query_database, 1);
482 assert_int_equal(get_customer_id_by_name(&connection, "john doe"), 543);
485 int main(int argc, char* argv[]) {
486 const UnitTest tests[] = {
487 unit_test(test_connect_to_customer_database),
488 unit_test(fail_connect_to_customer_database),
489 unit_test(test_get_customer_id_by_name),
491 return run_tests(tests);
495 <a name="MockFunctionsCheckingParameters"><h3>Checking Parameters</h3></a>
496 <p>In addition to storing the return values of mock functions, Cmockery
497 provides functionality to store expected values for mock function parameters
498 using the expect_*() functions provided. A mock function parameter can then
499 be validated using the check_expected() macro.
501 <p>Successive calls to expect_*() macros for a parameter queues values to
502 check the specified parameter. check_expected() checks a function parameter
503 against the next value queued using expect_*(), if the parameter check fails a
504 test failure is signalled. In addition if check_expected() is called and
505 no more parameter values are queued a test failure occurs.</p>
507 <h4>Using expect_*()</h4>
508 <a href="../src/example/product_database.c">product_database.c</a>
510 #include <database.h>
512 // Connect to the database containing customer information.
513 DatabaseConnection* connect_to_product_database() {
514 return connect_to_database("products.abcd.org", 322);
517 <a href="../src/example/product_database_test.c">product_database_test.c</a>
519 #include <stdarg.h>
520 #include <stddef.h>
521 #include <setjmp.h>
522 #include <cmockery.h>
523 #include <database.h>
525 extern DatabaseConnection* connect_to_product_database();
527 /* Mock connect to database function.
528 * NOTE: This mock function is very general could be shared between tests
529 * that use the imaginary database.h module. */
530 DatabaseConnection* connect_to_database(const char * const url,
531 const unsigned int port) {
533 check_expected(port);
534 return (DatabaseConnection*)mock();
537 void test_connect_to_product_database(void **state) {
538 expect_string(connect_to_database, url, "products.abcd.org");
539 expect_value(connect_to_database, port, 322);
540 will_return(connect_to_database, 0xDA7ABA53);
541 assert_int_equal(connect_to_product_database(), 0xDA7ABA53);
544 /* This test will fail since the expected URL is different to the URL that is
545 * passed to connect_to_database() by connect_to_product_database(). */
546 void test_connect_to_product_database_bad_url(void **state) {
547 expect_string(connect_to_database, url, "products.abcd.com");
548 expect_value(connect_to_database, port, 322);
549 will_return(connect_to_database, 0xDA7ABA53);
550 assert_int_equal((int)connect_to_product_database(), 0xDA7ABA53);
553 /* This test will fail since the mock connect_to_database() will attempt to
554 * retrieve a value for the parameter port which isn't specified by this
556 void test_connect_to_product_database_missing_parameter(void **state) {
557 expect_string(connect_to_database, url, "products.abcd.org");
558 will_return(connect_to_database, 0xDA7ABA53);
559 assert_int_equal((int)connect_to_product_database(), 0xDA7ABA53);
562 int main(int argc, char* argv[]) {
563 const UnitTest tests[] = {
564 unit_test(test_connect_to_product_database),
565 unit_test(test_connect_to_product_database_bad_url),
566 unit_test(test_connect_to_product_database_missing_parameter),
568 return run_tests(tests);
572 <a name="TestState"><h2>Test State</h2></a>
574 <p>Cmockery allows the specification of multiple setup and tear down functions
575 for each test case. Setup functions, specified by the <b>unit_test_setup()</b>
576 or <b>unit_test_setup_teardown()</b> macros allow common initialization to be
577 shared between multiple test cases. In addition, tear down functions,
578 specified by the <b>unit_test_teardown()</b> or
579 <b>unit_test_setup_teardown()</b> macros provide a code path that is always
580 executed for a test case even when it fails.</p>
582 <h4>Using unit_test_setup_teardown()</h4>
583 <a href="../src/example/key_value.c">key_value.c</a>
585 #include <stddef.h>
586 #include <stdlib.h>
587 #include <string.h>
589 typedef struct KeyValue {
594 static KeyValue *key_values = NULL;
595 static unsigned int number_of_key_values = 0;
597 void set_key_values(KeyValue * const new_key_values,
598 const unsigned int new_number_of_key_values) {
599 key_values = new_key_values;
600 number_of_key_values = new_number_of_key_values;
603 // Compare two key members of KeyValue structures.
604 int key_value_compare_keys(const void *a, const void *b) {
605 return (int)((KeyValue*)a)->key - (int)((KeyValue*)b)->key;
608 // Search an array of key value pairs for the item with the specified value.
609 KeyValue* find_item_by_value(const char * const value) {
611 for (i = 0; i < number_of_key_values; i++) {
612 if (strcmp(key_values[i].value, value) == 0) {
613 return &key_values[i];
619 // Sort an array of key value pairs by key.
620 void sort_items_by_key() {
621 qsort(key_values, number_of_key_values, sizeof(*key_values),
622 key_value_compare_keys);
625 <a href="../src/example/key_value_test.c">key_value_test.c</a>
627 #include <stdarg.h>
628 #include <stddef.h>
629 #include <setjmp.h>
630 #include <string.h>
631 #include <cmockery.h>
633 /* This is duplicated here from the module setup_teardown.c to reduce the
634 * number of files used in this test. */
635 typedef struct KeyValue {
640 void set_key_values(KeyValue * const new_key_values,
641 const unsigned int new_number_of_key_values);
642 extern KeyValue* find_item_by_value(const char * const value);
643 extern void sort_items_by_key();
645 static KeyValue key_values[] = {
652 void create_key_values(void **state) {
653 KeyValue * const items = (KeyValue*)test_malloc(sizeof(key_values));
654 memcpy(items, key_values, sizeof(key_values));
655 *state = (void*)items;
656 set_key_values(items, sizeof(key_values) / sizeof(key_values[0]));
659 void destroy_key_values(void **state) {
661 set_key_values(NULL, 0);
664 void test_find_item_by_value(void **state) {
666 for (i = 0; i < sizeof(key_values) / sizeof(key_values[0]); i++) {
667 KeyValue * const found = find_item_by_value(key_values[i].value);
669 assert_int_equal(found->key, key_values[i].key);
670 assert_string_equal(found->value, key_values[i].value);
674 void test_sort_items_by_key(void **state) {
676 KeyValue * const kv = *state;
678 for (i = 1; i < sizeof(key_values) / sizeof(key_values[0]); i++) {
679 assert_true(kv[i - 1].key < kv[i].key);
683 int main(int argc, char* argv[]) {
684 const UnitTest tests[] = {
685 unit_test_setup_teardown(test_find_item_by_value, create_key_values,
687 unit_test_setup_teardown(test_sort_items_by_key, create_key_values,
690 return run_tests(tests);
694 <a name="Example"><h2>Example</h2></a>
696 <p>A small command line calculator
697 <a href="../src/example/calculator.c">calculator.c</a> application
698 and test application that full exercises the calculator application
699 <a href="../src/example/calculator_test.c">calculator_test.c</a>
700 are provided as an example of Cmockery's features discussed in this document.
705 <!-- hhmts start --> Last modified: Mon Jul 20 15:57:27 PDT 2009 <!-- hhmts end -->