1 .. SPDX-License-Identifier: GPL-2.0
2 .. Copyright (C) 2019, Google LLC.
4 The Kernel Concurrency Sanitizer (KCSAN)
5 ========================================
7 The Kernel Concurrency Sanitizer (KCSAN) is a dynamic race detector, which
8 relies on compile-time instrumentation, and uses a watchpoint-based sampling
9 approach to detect races. KCSAN's primary purpose is to detect `data races`_.
14 KCSAN is supported by both GCC and Clang. With GCC we require version 11 or
15 later, and with Clang also require version 11 or later.
17 To enable KCSAN configure the kernel with::
21 KCSAN provides several other configuration options to customize behaviour (see
22 the respective help text in ``lib/Kconfig.kcsan`` for more info).
27 A typical data race report looks like this::
29 ==================================================================
30 BUG: KCSAN: data-race in test_kernel_read / test_kernel_write
32 write to 0xffffffffc009a628 of 8 bytes by task 487 on cpu 0:
33 test_kernel_write+0x1d/0x30
34 access_thread+0x89/0xd0
36 ret_from_fork+0x22/0x30
38 read to 0xffffffffc009a628 of 8 bytes by task 488 on cpu 6:
39 test_kernel_read+0x10/0x20
40 access_thread+0x89/0xd0
42 ret_from_fork+0x22/0x30
44 value changed: 0x00000000000009a6 -> 0x00000000000009b2
46 Reported by Kernel Concurrency Sanitizer on:
47 CPU: 6 PID: 488 Comm: access_thread Not tainted 5.12.0-rc2+ #1
48 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.14.0-2 04/01/2014
49 ==================================================================
51 The header of the report provides a short summary of the functions involved in
52 the race. It is followed by the access types and stack traces of the 2 threads
53 involved in the data race. If KCSAN also observed a value change, the observed
54 old value and new value are shown on the "value changed" line respectively.
56 The other less common type of data race report looks like this::
58 ==================================================================
59 BUG: KCSAN: data-race in test_kernel_rmw_array+0x71/0xd0
61 race at unknown origin, with read to 0xffffffffc009bdb0 of 8 bytes by task 515 on cpu 2:
62 test_kernel_rmw_array+0x71/0xd0
63 access_thread+0x89/0xd0
65 ret_from_fork+0x22/0x30
67 value changed: 0x0000000000002328 -> 0x0000000000002329
69 Reported by Kernel Concurrency Sanitizer on:
70 CPU: 2 PID: 515 Comm: access_thread Not tainted 5.12.0-rc2+ #1
71 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.14.0-2 04/01/2014
72 ==================================================================
74 This report is generated where it was not possible to determine the other
75 racing thread, but a race was inferred due to the data value of the watched
76 memory location having changed. These reports always show a "value changed"
77 line. A common reason for reports of this type are missing instrumentation in
78 the racing thread, but could also occur due to e.g. DMA accesses. Such reports
79 are shown only if ``CONFIG_KCSAN_REPORT_RACE_UNKNOWN_ORIGIN=y``, which is
85 It may be desirable to disable data race detection for specific accesses,
86 functions, compilation units, or entire subsystems. For static blacklisting,
87 the below options are available:
89 * KCSAN understands the ``data_race(expr)`` annotation, which tells KCSAN that
90 any data races due to accesses in ``expr`` should be ignored and resulting
91 behaviour when encountering a data race is deemed safe. Please see
92 `"Marking Shared-Memory Accesses" in the LKMM`_ for more information.
94 * Disabling data race detection for entire functions can be accomplished by
95 using the function attribute ``__no_kcsan``::
101 To dynamically limit for which functions to generate reports, see the
102 `DebugFS interface`_ blacklist/whitelist feature.
104 * To disable data race detection for a particular compilation unit, add to the
107 KCSAN_SANITIZE_file.o := n
109 * To disable data race detection for all compilation units listed in a
110 ``Makefile``, add to the respective ``Makefile``::
114 .. _"Marking Shared-Memory Accesses" in the LKMM: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/memory-model/Documentation/access-marking.txt
116 Furthermore, it is possible to tell KCSAN to show or hide entire classes of
117 data races, depending on preferences. These can be changed via the following
120 * ``CONFIG_KCSAN_REPORT_VALUE_CHANGE_ONLY``: If enabled and a conflicting write
121 is observed via a watchpoint, but the data value of the memory location was
122 observed to remain unchanged, do not report the data race.
124 * ``CONFIG_KCSAN_ASSUME_PLAIN_WRITES_ATOMIC``: Assume that plain aligned writes
125 up to word size are atomic by default. Assumes that such writes are not
126 subject to unsafe compiler optimizations resulting in data races. The option
127 causes KCSAN to not report data races due to conflicts where the only plain
128 accesses are aligned writes up to word size.
133 The file ``/sys/kernel/debug/kcsan`` provides the following interface:
135 * Reading ``/sys/kernel/debug/kcsan`` returns various runtime statistics.
137 * Writing ``on`` or ``off`` to ``/sys/kernel/debug/kcsan`` allows turning KCSAN
138 on or off, respectively.
140 * Writing ``!some_func_name`` to ``/sys/kernel/debug/kcsan`` adds
141 ``some_func_name`` to the report filter list, which (by default) blacklists
142 reporting data races where either one of the top stackframes are a function
145 * Writing either ``blacklist`` or ``whitelist`` to ``/sys/kernel/debug/kcsan``
146 changes the report filtering behaviour. For example, the blacklist feature
147 can be used to silence frequently occurring data races; the whitelist feature
148 can help with reproduction and testing of fixes.
153 Core parameters that affect KCSAN's overall performance and bug detection
154 ability are exposed as kernel command-line arguments whose defaults can also be
155 changed via the corresponding Kconfig options.
157 * ``kcsan.skip_watch`` (``CONFIG_KCSAN_SKIP_WATCH``): Number of per-CPU memory
158 operations to skip, before another watchpoint is set up. Setting up
159 watchpoints more frequently will result in the likelihood of races to be
160 observed to increase. This parameter has the most significant impact on
161 overall system performance and race detection ability.
163 * ``kcsan.udelay_task`` (``CONFIG_KCSAN_UDELAY_TASK``): For tasks, the
164 microsecond delay to stall execution after a watchpoint has been set up.
165 Larger values result in the window in which we may observe a race to
168 * ``kcsan.udelay_interrupt`` (``CONFIG_KCSAN_UDELAY_INTERRUPT``): For
169 interrupts, the microsecond delay to stall execution after a watchpoint has
170 been set up. Interrupts have tighter latency requirements, and their delay
171 should generally be smaller than the one chosen for tasks.
173 They may be tweaked at runtime via ``/sys/module/kcsan/parameters/``.
178 In an execution, two memory accesses form a *data race* if they *conflict*,
179 they happen concurrently in different threads, and at least one of them is a
180 *plain access*; they *conflict* if both access the same memory location, and at
181 least one is a write. For a more thorough discussion and definition, see `"Plain
182 Accesses and Data Races" in the LKMM`_.
184 .. _"Plain Accesses and Data Races" in the LKMM: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/memory-model/Documentation/explanation.txt#n1922
186 Relationship with the Linux-Kernel Memory Consistency Model (LKMM)
187 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
189 The LKMM defines the propagation and ordering rules of various memory
190 operations, which gives developers the ability to reason about concurrent code.
191 Ultimately this allows to determine the possible executions of concurrent code,
192 and if that code is free from data races.
194 KCSAN is aware of *marked atomic operations* (``READ_ONCE``, ``WRITE_ONCE``,
195 ``atomic_*``, etc.), but is oblivious of any ordering guarantees and simply
196 assumes that memory barriers are placed correctly. In other words, KCSAN
197 assumes that as long as a plain access is not observed to race with another
198 conflicting access, memory operations are correctly ordered.
200 This means that KCSAN will not report *potential* data races due to missing
201 memory ordering. Developers should therefore carefully consider the required
202 memory ordering requirements that remain unchecked. If, however, missing
203 memory ordering (that is observable with a particular compiler and
204 architecture) leads to an observable data race (e.g. entering a critical
205 section erroneously), KCSAN would report the resulting data race.
207 Race Detection Beyond Data Races
208 --------------------------------
210 For code with complex concurrency design, race-condition bugs may not always
211 manifest as data races. Race conditions occur if concurrently executing
212 operations result in unexpected system behaviour. On the other hand, data races
213 are defined at the C-language level. The following macros can be used to check
214 properties of concurrent code where bugs would not manifest as data races.
216 .. kernel-doc:: include/linux/kcsan-checks.h
217 :functions: ASSERT_EXCLUSIVE_WRITER ASSERT_EXCLUSIVE_WRITER_SCOPED
218 ASSERT_EXCLUSIVE_ACCESS ASSERT_EXCLUSIVE_ACCESS_SCOPED
219 ASSERT_EXCLUSIVE_BITS
221 Implementation Details
222 ----------------------
224 KCSAN relies on observing that two accesses happen concurrently. Crucially, we
225 want to (a) increase the chances of observing races (especially for races that
226 manifest rarely), and (b) be able to actually observe them. We can accomplish
227 (a) by injecting various delays, and (b) by using address watchpoints (or
230 If we deliberately stall a memory access, while we have a watchpoint for its
231 address set up, and then observe the watchpoint to fire, two accesses to the
232 same address just raced. Using hardware watchpoints, this is the approach taken
234 <http://usenix.org/legacy/events/osdi10/tech/full_papers/Erickson.pdf>`_.
235 Unlike DataCollider, KCSAN does not use hardware watchpoints, but instead
236 relies on compiler instrumentation and "soft watchpoints".
238 In KCSAN, watchpoints are implemented using an efficient encoding that stores
239 access type, size, and address in a long; the benefits of using "soft
240 watchpoints" are portability and greater flexibility. KCSAN then relies on the
241 compiler instrumenting plain accesses. For each instrumented plain access:
243 1. Check if a matching watchpoint exists; if yes, and at least one access is a
244 write, then we encountered a racing access.
246 2. Periodically, if no matching watchpoint exists, set up a watchpoint and
247 stall for a small randomized delay.
249 3. Also check the data value before the delay, and re-check the data value
250 after delay; if the values mismatch, we infer a race of unknown origin.
252 To detect data races between plain and marked accesses, KCSAN also annotates
253 marked accesses, but only to check if a watchpoint exists; i.e. KCSAN never
254 sets up a watchpoint on marked accesses. By never setting up watchpoints for
255 marked operations, if all accesses to a variable that is accessed concurrently
256 are properly marked, KCSAN will never trigger a watchpoint and therefore never
262 1. **Memory Overhead:** The overall memory overhead is only a few MiB
263 depending on configuration. The current implementation uses a small array of
264 longs to encode watchpoint information, which is negligible.
266 2. **Performance Overhead:** KCSAN's runtime aims to be minimal, using an
267 efficient watchpoint encoding that does not require acquiring any shared
268 locks in the fast-path. For kernel boot on a system with 8 CPUs:
270 - 5.0x slow-down with the default KCSAN config;
271 - 2.8x slow-down from runtime fast-path overhead only (set very large
272 ``KCSAN_SKIP_WATCH`` and unset ``KCSAN_SKIP_WATCH_RANDOMIZE``).
274 3. **Annotation Overheads:** Minimal annotations are required outside the KCSAN
275 runtime. As a result, maintenance overheads are minimal as the kernel
278 4. **Detects Racy Writes from Devices:** Due to checking data values upon
279 setting up watchpoints, racy writes from devices can also be detected.
281 5. **Memory Ordering:** KCSAN is *not* explicitly aware of the LKMM's ordering
282 rules; this may result in missed data races (false negatives).
284 6. **Analysis Accuracy:** For observed executions, due to using a sampling
285 strategy, the analysis is *unsound* (false negatives possible), but aims to
286 be complete (no false positives).
288 Alternatives Considered
289 -----------------------
291 An alternative data race detection approach for the kernel can be found in the
292 `Kernel Thread Sanitizer (KTSAN) <https://github.com/google/ktsan/wiki>`_.
293 KTSAN is a happens-before data race detector, which explicitly establishes the
294 happens-before order between memory operations, which can then be used to
295 determine data races as defined in `Data Races`_.
297 To build a correct happens-before relation, KTSAN must be aware of all ordering
298 rules of the LKMM and synchronization primitives. Unfortunately, any omission
299 leads to large numbers of false positives, which is especially detrimental in
300 the context of the kernel which includes numerous custom synchronization
301 mechanisms. To track the happens-before relation, KTSAN's implementation
302 requires metadata for each memory location (shadow memory), which for each page
303 corresponds to 4 pages of shadow memory, and can translate into overhead of
304 tens of GiB on a large system.