1 <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
3 <title>Cmockery</title>
6 <h1>Cmockery Unit Testing Framework</h1>
7 <p>Cmockery is a lightweight library that is used to author C unit tests.</p>
10 <li><a href="#Motivation">Motivation</a></li>
11 <li><a href="#Overview">Overview</a></li>
12 <li><a href="#TestExecution">Test Execution</a>
13 <li><a href="#ExceptionHandling">Exception Handling</a></li>
14 <li><a href="#FailureConditions">Failure Conditions</a></li>
15 <li><a href="#Assertions">Assertions</a></li>
17 <li><a href="#AssertMacros">Assert Macros</a></li>
19 <li><a href="#MemoryAllocation">Dynamic Memory Allocation</a></li>
20 <li><a href="#MockFunctions">Mock functions</a></li>
22 <li><a href="#MockFunctionsReturnValues">Return Values</a></li>
23 <li><a href="#MockFunctionsCheckingParameters">Checking Parameters</a></li>
25 <li><a href="#TestState">Test State</a></li>
26 <li><a href="#Example">Example</a></li>
29 <a name="Motivation"><h2>Motivation</h2></a>
30 <p>There are a variety of C unit testing frameworks available however many of
31 them are fairly complex and require the latest compiler technology. Some
32 development requires the use of old compilers which makes it difficult to
33 use some unit testing frameworks. In addition many unit testing frameworks
34 assume the code being tested is an application or module that is targeted to
35 the same platform that will ultimately execute the test. Because of this
36 assumption many frameworks require the inclusion of standard C library headers
37 in the code module being tested which may collide with the custom or
38 incomplete implementation of the C library utilized by the code under test.</p>
40 <p>Cmockery only requires a test application is linked with the standard C
41 library which minimizes conflicts with standard C library headers. Also,
42 Cmockery tries avoid the use of some of the newer features of C compilers.</p>
44 <p>This results in Cmockery being a relatively small library that can be used
45 to test a variety of exotic code. If a developer wishes to simply test an
46 application with the latest compiler then other unit testing frameworks maybe
49 <a name="Overview"><h2>Overview</h2></a>
50 <p>Cmockery tests are compiled into stand-alone executables and linked with
51 the Cmockery library, the standard C library and module being tested. Any
52 symbols external to the module being tested should be mocked - replaced with
53 functions that return values determined by the test - within the test
54 application. Even though significant differences may exist between the target
55 execution environment of a code module and the environment used to test the
56 code the unit testing is still valid since its goal is to test the logic of a
57 code modules at a functional level and not necessarily all of its interactions
58 with the target execution environment.</p>
60 <p>It may not be possible to compile a module into a test application without
61 some modification, therefore the preprocessor symbol <b>UNIT_TESTING</b> should
62 be defined when Cmockery unit test applications are compiled so code within the
63 module can be conditionally compiled for tests.</p>
65 <a name="TestExecution"><h2>Test Execution</h2></a>
66 <p>Cmockery unit test cases are functions with the signature
67 <b>void function(void **state)</b>. Cmockery test applications initialize a
68 table with test case function pointers using <b>unit_test*()</b> macros. This
69 table is then passed to the <b>run_tests()</b> macro to execute the tests.
71 <b>run_tests()</b> sets up the appropriate exception / signal handlers and
72 other data structures prior to running each test function. When a unit test
73 is complete <b>run_tests()</b> performs various checks to determine whether
74 the test succeeded.</p>
76 <h4>Using run_tests()</h4>
78 <a href="../src/example/run_tests.c">run_tests.c</a>
79 -------------------------------------------------------------------------------
80 #include <stdarg.h>
81 #include <stddef.h>
82 #include <setjmp.h>
83 #include <cmockery.h>
85 // A test case that does nothing and succeeds.
86 void null_test_success(void **state) {
89 int main(int argc, char* argv[]) {
90 const UnitTest tests[] = {
91 unit_test(null_test_success),
93 return run_tests(tests);
97 <a name="ExceptionHandling"><h2>Exception Handling</h2></a>
98 <p>Before a test function is executed by <b>run_tests()</b>,
99 exception / signal handlers are overridden with a handler that simply
100 displays an error and exits a test function if an exception occurs. If an
101 exception occurs outside of a test function, for example in Cmockery itself,
102 the application aborts execution and returns an error code.</p>
104 <a name="FailureConditions"><h2>Failure Conditions</h2></a>
105 <p>If a failure occurs during a test function that's executed via
106 <b>run_tests()</b>, the test function is aborted and the application's
107 execution resumes with the next test function.
109 Test failures are ultimately signalled via the Cmockery function <b>fail()</b>.
110 The following events will result in the Cmockery library signalling a test
114 <li><a href="#Assertions">Assertions</a></li>
115 <li><a href="#ExceptionHandling">Exceptions</a></li>
116 <li><a href="#MemoryAllocation">Memory leaks</a></li>
117 <li><a href="#TestState">Mismatched setup and tear down functions</a></li>
118 <li><a href="#MockFunctionsReturnValues">Missing mock return values</a></li>
119 <li><a href="#MockFunctionsReturnValues">Unused mock return values</a></li>
120 <li><a href="#MockFunctionsCheckingParameters">Missing expected parameter
122 <li><a href="#MockFunctionsCheckingParameters">Unused expected parameter
127 <a name="Assertions"><h2>Assertions</h2></a>
128 <p>Runtime assert macros like the standard C library's <b>assert()</b> should
129 be redefined in modules being tested to use Cmockery's <b>mock_assert()</b>
130 function. Normally <b>mock_assert()</b> signals a
131 <a href="#FailureConditions">test failure</a>. If a function is called using
132 the <b>expect_assert_failure()</b> macro, any calls to <b>mock_assert()</b>
133 within the function will result in the execution of the test. If no
134 calls to <b>mock_assert()</b> occur during the function called via
135 <b>expect_assert_failure()</b> a test failure is signalled.</p>
137 <h4>Using mock_assert()</h4>
139 <a href="../src/example/assert_module.c">assert_module.c</a>
140 -------------------------------------------------------------------------------
141 #include <assert.h>
143 // If unit testing is enabled override assert with mock_assert().
145 extern void mock_assert(const int result, const char* const expression,
146 const char * const file, const int line);
148 #define assert(expression) \
149 mock_assert((int)(expression), #expression, __FILE__, __LINE__);
150 #endif // UNIT_TESTING
152 void increment_value(int * const value) {
157 void decrement_value(int * const value) {
163 <a href="../src/example/assert_module_test.c">assert_module_test.c</a>
164 -------------------------------------------------------------------------------
165 #include <stdarg.h>
166 #include <stddef.h>
167 #include <setjmp.h>
168 #include <cmockery.h>
170 extern void increment_value(int * const value);
172 /* This test case will fail but the assert is caught by run_tests() and the
173 * next test is executed. */
174 void increment_value_fail(void **state) {
175 increment_value(NULL);
178 // This test case succeeds since increment_value() asserts on the NULL pointer.
179 void increment_value_assert(void **state) {
180 expect_assert_failure(increment_value(NULL));
183 /* This test case fails since decrement_value() doesn't assert on a NULL
185 void decrement_value_fail(void **state) {
186 expect_assert_failure(decrement_value(NULL));
189 int main(int argc, char *argv[]) {
190 const UnitTest tests[] = {
191 unit_test(increment_value_fail),
192 unit_test(increment_value_assert),
193 unit_test(decrement_value_fail),
195 return run_tests(tests);
199 <h3><a name="AssertMacros">Assert Macros</a></h3>
201 <p>Cmockery provides an assortment of assert macros that tests applications
202 should use use in preference to the C standard library's assert macro. On an
203 assertion failure a Cmockery assert macro will write the failure to the
204 standard error stream and signal a test failure. Due to limitations of the
205 C language the general C standard library assert() and Cmockery's
206 assert_true() and assert_false() macros can only display the expression that
207 caused the assert failure. Cmockery's type specific assert macros,
208 assert_{type}_equal() and assert_{type}_not_equal(), display the data that
209 caused the assertion failure which increases data visibility aiding
210 debugging of failing test cases.</p>
212 <h4>Using assert_{type}_equal() macros</h4>
214 <a href="../src/example/assert_macro.c">assert_macro.c</a>
215 -------------------------------------------------------------------------------
216 #include <string.h>
218 static const char* status_code_strings[] = {
220 "Connection dropped",
221 "Connection timed out",
224 const char* get_status_code_string(const unsigned int status_code) {
225 return status_code_strings[status_code];
228 unsigned int string_to_status_code(const char* const status_code_string) {
230 for (i = 0; i < sizeof(status_code_string) / sizeof(status_code_string[0]);
232 if (strcmp(status_code_strings[i], status_code_string) == 0) {
239 <a href="../src/example/assert_macro_test.c">assert_macro_test.c</a>
240 -------------------------------------------------------------------------------
241 #include <stdarg.h>
242 #include <stddef.h>
243 #include <setjmp.h>
244 #include <cmockery.h>
246 extern const char* get_status_code_string(const unsigned int status_code);
247 extern unsigned int string_to_status_code(
248 const char* const status_code_string);
250 /* This test will fail since the string returned by get_status_code_string(0)
251 * doesn't match "Connection timed out". */
252 void get_status_code_string_test(void **state) {
253 assert_string_equal(get_status_code_string(0), "Address not found");
254 assert_string_equal(get_status_code_string(1), "Connection timed out");
257 // This test will fail since the status code of "Connection timed out" isn't 1
258 void string_to_status_code_test(void **state) {
259 assert_int_equal(string_to_status_code("Address not found"), 0);
260 assert_int_equal(string_to_status_code("Connection timed out"), 1);
263 int main(int argc, char *argv[]) {
264 const UnitTest tests[] = {
265 unit_test(get_status_code_string_test),
266 unit_test(string_to_status_code_test),
268 return run_tests(tests);
272 <a name="MemoryAllocation"><h2>Dynamic Memory Allocation</h2></a>
274 <p>To test for memory leaks, buffer overflows and underflows a module being
275 tested by Cmockery should replace calls to <b>malloc()</b>, <b>calloc()</b> and
276 <b>free()</b> to <b>test_malloc()</b>, <b>test_calloc()</b> and
277 <b>test_free()</b> respectively. Each time a block is deallocated using
278 <b>test_free()</b> it is checked for corruption, if a corrupt block is found
279 a <a href="#FailureConditions">test failure</a> is signalled. All blocks
280 allocated using the <b>test_*()</b> allocation functions are tracked by the
281 Cmockery library. When a test completes if any allocated blocks (memory leaks)
282 remain they are reported and a test failure is signalled.</p>
283 <p>For simplicity Cmockery currently executes all tests in one process.
284 Therefore all test cases in a test application share a single address space
285 which means memory corruption from a single test case could potentially cause
286 the test application to exit prematurely.</p>
288 <h4>Using Cmockery's Allocators</h4>
290 <a href="../src/example/allocate_module.c">allocate_module.c</a>
291 -------------------------------------------------------------------------------
292 #include <malloc.h>
295 extern void* _test_malloc(const size_t size, const char* file, const int line);
296 extern void* _test_calloc(const size_t number_of_elements, const size_t size,
297 const char* file, const int line);
298 extern void _test_free(void* const ptr, const char* file, const int line);
300 #define malloc(size) _test_malloc(size, __FILE__, __LINE__)
301 #define calloc(num, size) _test_calloc(num, size, __FILE__, __LINE__)
302 #define free(ptr) _test_free(ptr, __FILE__, __LINE__)
303 #endif // UNIT_TESTING
306 int * const temporary = (int*)malloc(sizeof(int));
310 void buffer_overflow() {
311 char * const memory = (char*)malloc(sizeof(int));
312 memory[sizeof(int)] = '!';
316 void buffer_underflow() {
317 char * const memory = (char*)malloc(sizeof(int));
323 <a href="../src/example/allocate_module_test.c">allocate_module_test.c</a>
324 -------------------------------------------------------------------------------
325 #include <stdarg.h>
326 #include <stddef.h>
327 #include <setjmp.h>
328 #include <cmockery.h>
330 extern void leak_memory();
331 extern void buffer_overflow();
332 extern void buffer_underflow();
334 // Test case that fails as leak_memory() leaks a dynamically allocated block.
335 void leak_memory_test(void **state) {
339 // Test case that fails as buffer_overflow() corrupts an allocated block.
340 void buffer_overflow_test(void **state) {
344 // Test case that fails as buffer_underflow() corrupts an allocated block.
345 void buffer_underflow_test(void **state) {
349 int main(int argc, char* argv[]) {
350 const UnitTest tests[] = {
351 unit_test(leak_memory_test),
352 unit_test(buffer_overflow_test),
353 unit_test(buffer_underflow_test),
355 return run_tests(tests);
359 <a name="MockFunctions"><h2>Mock Functions</h2></a>
361 <p>A unit test should ideally isolate the function or module being tested
362 from any external dependencies. This can be performed using mock functions
363 that are either statically or dynamically linked with the module being tested.
364 Mock functions must be statically linked when the code being tested directly
365 references external functions. Dynamic linking is simply the process of
366 setting a function pointer in a table used by the tested module to reference
367 a mock function defined in the unit test.</p>
369 <a name="MockFunctionsReturnValues"><h3>Return Values</h3></a>
371 <p>In order to simplify the implementation of mock functions Cmockery provides
372 functionality which stores return values for mock functions in each test
373 case using <b>will_return()</b>. These values are then returned by each mock
374 function using calls to <b>mock()</b>.
376 Values passed to <b>will_return()</b> are added to a queue for each function
377 specified. Each successive call to <b>mock()</b> from a function removes a
378 return value from the queue. This makes it possible for a mock function to use
379 multiple calls to <b>mock()</b> to return output parameters in addition to a
380 return value. In addition this allows the specification of return values for
381 multiple calls to a mock function.</p>
383 <h4>Using will_return()</h4>
385 <a name="../src/example/database.h" href="database.h">database.h</a>
386 -------------------------------------------------------------------------------
387 typedef struct DatabaseConnection DatabaseConnection;
389 /* Function that takes an SQL query string and sets results to an array of
390 * pointers with the result of the query. The value returned specifies the
391 * number of items in the returned array of results. The returned array of
392 * results are statically allocated and should not be deallocated using free()
394 typedef unsigned int (*QueryDatabase)(
395 DatabaseConnection* const connection, const char * const query_string,
396 void *** const results);
398 // Connection to a database.
399 struct DatabaseConnection {
402 QueryDatabase query_database;
405 // Connect to a database.
406 DatabaseConnection* connect_to_database(const char * const url,
407 const unsigned int port);
410 <a href="../src/example/customer_database.c">customer_database.c</a>
411 -------------------------------------------------------------------------------
412 #include <stddef.h>
413 #include <stdio.h>
414 #include <a href="#database.h"><database.h></a>
416 #define snprintf _snprintf
419 // Connect to the database containing customer information.
420 DatabaseConnection* connect_to_customer_database() {
421 return connect_to_database("customers.abcd.org", 321);
424 /* Find the ID of a customer by his/her name returning a value > 0 if
425 * successful, 0 otherwise. */
426 unsigned int get_customer_id_by_name(
427 DatabaseConnection * const connection,
428 const char * const customer_name) {
429 char query_string[256];
430 int number_of_results;
432 snprintf(query_string, sizeof(query_string),
433 "SELECT ID FROM CUSTOMERS WHERE NAME = %s", customer_name);
434 number_of_results = connection->query_database(connection, query_string,
436 if (number_of_results != 1) {
439 return (unsigned int)results[0];
443 <a href="../src/example/customer_database_test.c">customer_database_test.c</a>
444 -------------------------------------------------------------------------------
445 #include <stdarg.h>
446 #include <stddef.h>
447 #include <setjmp.h>
448 #include <cmockery.h>
449 #include <a href="#database.h"><database.h></a>
452 extern DatabaseConnection* connect_to_customer_database();
453 extern unsigned int get_customer_id_by_name(
454 DatabaseConnection * const connection, const char * const customer_name);
456 // Mock query database function.
457 unsigned int mock_query_database(
458 DatabaseConnection* const connection, const char * const query_string,
459 void *** const results) {
460 *results = (void**)mock();
461 return (unsigned int)mock();
464 // Mock of the connect to database function.
465 DatabaseConnection* connect_to_database(const char * const database_url,
466 const unsigned int port) {
467 return (DatabaseConnection*)mock();
470 void test_connect_to_customer_database(void **state) {
471 will_return(connect_to_database, 0x0DA7ABA53);
472 assert_true(connect_to_customer_database() ==
473 (DatabaseConnection*)0x0DA7ABA53);
476 /* This test fails as the mock function connect_to_database() will have no
477 * value to return. */
478 void fail_connect_to_customer_database(void **state) {
479 will_return(connect_to_database, 0x0DA7ABA53);
480 assert_true(connect_to_customer_database() ==
481 (DatabaseConnection*)0x0DA7ABA53);
484 void test_get_customer_id_by_name(void **state) {
485 DatabaseConnection connection = {
486 "somedatabase.somewhere.com", 12345678, mock_query_database
488 // Return a single customer ID when mock_query_database() is called.
489 int customer_ids = 543;
490 will_return(mock_query_database, &customer_ids);
491 will_return(mock_query_database, 1);
492 assert_int_equal(get_customer_id_by_name(&connection, "john doe"), 543);
495 int main(int argc, char* argv[]) {
496 const UnitTest tests[] = {
497 unit_test(test_connect_to_customer_database),
498 unit_test(fail_connect_to_customer_database),
499 unit_test(test_get_customer_id_by_name),
501 return run_tests(tests);
505 <a name="MockFunctionsCheckingParameters"><h3>Checking Parameters</h3></a>
506 <p>In addition to storing the return values of mock functions, Cmockery
507 provides functionality to store expected values for mock function parameters
508 using the expect_*() functions provided. A mock function parameter can then
509 be validated using the check_expected() macro.
511 <p>Successive calls to expect_*() macros for a parameter queues values to
512 check the specified parameter. check_expected() checks a function parameter
513 against the next value queued using expect_*(), if the parameter check fails a
514 test failure is signalled. In addition if check_expected() is called and
515 no more parameter values are queued a test failure occurs.</p>
517 <h4>Using expect_*()</h4>
519 <a href="../src/example/product_database.c">product_database.c</a>
520 -------------------------------------------------------------------------------
521 #include <a href="#database.h"><database.h></a>
523 // Connect to the database containing customer information.
524 DatabaseConnection* connect_to_product_database() {
525 return connect_to_database("products.abcd.org", 322);
528 <a href="../src/example/product_database_test.c">product_database_test.c</a>
529 -------------------------------------------------------------------------------
530 #include <stdarg.h>
531 #include <stddef.h>
532 #include <setjmp.h>
533 #include <cmockery.h>
534 #include <a href="#database.h"><database.h></a>
536 extern DatabaseConnection* connect_to_product_database();
538 /* Mock connect to database function.
539 * NOTE: This mock function is very general could be shared between tests
540 * that use the imaginary database.h module. */
541 DatabaseConnection* connect_to_database(const char * const url,
542 const unsigned int port) {
544 check_expected(port);
545 return (DatabaseConnection*)mock();
548 void test_connect_to_product_database(void **state) {
549 expect_string(connect_to_database, url, "products.abcd.org");
550 expect_value(connect_to_database, port, 322);
551 will_return(connect_to_database, 0xDA7ABA53);
552 assert_int_equal(connect_to_product_database(), 0xDA7ABA53);
555 /* This test will fail since the expected URL is different to the URL that is
556 * passed to connect_to_database() by connect_to_product_database(). */
557 void test_connect_to_product_database_bad_url(void **state) {
558 expect_string(connect_to_database, url, "products.abcd.com");
559 expect_value(connect_to_database, port, 322);
560 will_return(connect_to_database, 0xDA7ABA53);
561 assert_int_equal((int)connect_to_product_database(), 0xDA7ABA53);
564 /* This test will fail since the mock connect_to_database() will attempt to
565 * retrieve a value for the parameter port which isn't specified by this
567 void test_connect_to_product_database_missing_parameter(void **state) {
568 expect_string(connect_to_database, url, "products.abcd.org");
569 will_return(connect_to_database, 0xDA7ABA53);
570 assert_int_equal((int)connect_to_product_database(), 0xDA7ABA53);
573 int main(int argc, char* argv[]) {
574 const UnitTest tests[] = {
575 unit_test(test_connect_to_product_database),
576 unit_test(test_connect_to_product_database_bad_url),
577 unit_test(test_connect_to_product_database_missing_parameter),
579 return run_tests(tests);
583 <a name="TestState"><h2>Test State</h2></a>
585 <p>Cmockery allows the specification of multiple setup and tear down functions
586 for each test case. Setup functions, specified by the <b>unit_test_setup()</b>
587 or <b>unit_test_setup_teardown()</b> macros allow common initialization to be
588 shared between multiple test cases. In addition, tear down functions,
589 specified by the <b>unit_test_teardown()</b> or
590 <b>unit_test_setup_teardown()</b> macros provide a code path that is always
591 executed for a test case even when it fails.</p>
593 <h4>Using unit_test_setup_teardown()</h4>
595 <a href="../src/example/key_value.c">key_value.c</a>
596 -------------------------------------------------------------------------------
597 #include <stddef.h>
598 #include <stdlib.h>
599 #include <string.h>
601 typedef struct KeyValue {
606 static KeyValue *key_values = NULL;
607 static unsigned int number_of_key_values = 0;
609 void set_key_values(KeyValue * const new_key_values,
610 const unsigned int new_number_of_key_values) {
611 key_values = new_key_values;
612 number_of_key_values = new_number_of_key_values;
615 // Compare two key members of KeyValue structures.
616 int key_value_compare_keys(const void *a, const void *b) {
617 return (int)((KeyValue*)a)->key - (int)((KeyValue*)b)->key;
620 // Search an array of key value pairs for the item with the specified value.
621 KeyValue* find_item_by_value(const char * const value) {
623 for (i = 0; i < number_of_key_values; i++) {
624 if (strcmp(key_values[i].value, value) == 0) {
625 return &key_values[i];
631 // Sort an array of key value pairs by key.
632 void sort_items_by_key() {
633 qsort(key_values, number_of_key_values, sizeof(*key_values),
634 key_value_compare_keys);
637 <a href="../src/example/key_value_test.c">key_value_test.c</a>
638 -------------------------------------------------------------------------------
639 #include <stdarg.h>
640 #include <stddef.h>
641 #include <setjmp.h>
642 #include <string.h>
643 #include <cmockery.h>
645 /* This is duplicated here from the module setup_teardown.c to reduce the
646 * number of files used in this test. */
647 typedef struct KeyValue {
652 void set_key_values(KeyValue * const new_key_values,
653 const unsigned int new_number_of_key_values);
654 extern KeyValue* find_item_by_value(const char * const value);
655 extern void sort_items_by_key();
657 static KeyValue key_values[] = {
664 void create_key_values(void **state) {
665 KeyValue * const items = (KeyValue*)test_malloc(sizeof(key_values));
666 memcpy(items, key_values, sizeof(key_values));
667 *state = (void*)items;
668 set_key_values(items, sizeof(key_values) / sizeof(key_values[0]));
671 void destroy_key_values(void **state) {
673 set_key_values(NULL, 0);
676 void test_find_item_by_value(void **state) {
678 for (i = 0; i < sizeof(key_values) / sizeof(key_values[0]); i++) {
679 KeyValue * const found = find_item_by_value(key_values[i].value);
681 assert_int_equal(found->key, key_values[i].key);
682 assert_string_equal(found->value, key_values[i].value);
686 void test_sort_items_by_key(void **state) {
688 KeyValue * const kv = *state;
690 for (i = 1; i < sizeof(key_values) / sizeof(key_values[0]); i++) {
691 assert_true(kv[i - 1].key < kv[i].key);
695 int main(int argc, char* argv[]) {
696 const UnitTest tests[] = {
697 unit_test_setup_teardown(test_find_item_by_value, create_key_values,
699 unit_test_setup_teardown(test_sort_items_by_key, create_key_values,
702 return run_tests(tests);
706 <a name="Example"><h2>Example</h2></a>
708 <p>A small command line calculator
709 <a href="../src/example/calculator.c">calculator.c</a> application
710 and test application that full exercises the calculator application
711 <a href="../src/example/calculator_test.c">calculator_test.c</a>
712 are provided as an example of Cmockery's features discussed in this document.
717 <!-- hhmts start --> Last modified: Tue Aug 26 09:33:31 PDT 2008 <!-- hhmts end -->