1 .. _docs-module-structure:
6 The Pigweed module structure is designed to keep as much code as possible for a
7 particular slice of functionality in one place. That means including the code
8 from multiple languages, as well as all the related documentation and tests.
10 Additionally, the structure is designed to limit the number of places a file
11 could go, so that when reading callsites it is obvious where a header is from.
12 That is where the duplicated ``<module>`` occurrences in file paths comes from.
14 Example module structure
15 ------------------------
16 .. code-block:: python
20 docs.rst # If there is just 1 docs file, call it docs.rst
21 README.md # All modules must have a short README for gittiles
23 BUILD.gn # GN build required
24 BUILD # Bazel build required
26 # C++ public headers; the repeated module name is required
30 # Exposed private headers go under internal/
31 public/pw_foo/internal/bar.h
32 public/pw_foo/internal/qux.h
34 # Public override headers must go in 'public_overrides'
35 public_overrides/gtest/gtest.h
36 public_overrides/string.h
38 # Private headers go into <module>_*/...
43 # C++ implementations go in the root
52 # C++ tests also go in the root
57 # Python files go into 'py/<module>/...'
58 py/setup.py # All Python must be a Python module with setup.py
59 py/foo_test.py # Tests go in py/ but outside of the Python module
65 # Go files go into 'go/...'
68 # Examples go in examples/, mixing different languages
75 # Size reports go under size_report/
78 size_report/use_case_a.cc
79 size_report/use_case_b.cc
81 # Protobuf definition files go into <module>_protos/...
82 pw_foo_protos/foo.proto
83 pw_foo_protos/internal/zap.proto
85 # Other directories are fine, but should be private.
88 collection_of_tests/...
89 code_relating_to_subfeature/...
93 Pigweed upstream modules are always named with a prefix ``pw_`` to enforce
94 namespacing. Projects using Pigweed that wish to make their own modules can use
95 whatever name they like, but we suggest picking a short prefix to namespace
96 your product (e.g. for an Internet of Toast project, perhaps the prefix could
104 Located ``{pw_module_dir}/public/<module>``. These are headers that must be
105 exposed due to C++ limitations (i.e. are included from the public interface,
106 but are not intended for public use).
108 **Public headers** should take the form:
110 ``{pw_module_dir}/public/<module>/*.h``
112 **Exposed private headers** should take the form:
114 ``{pw_module_dir}/public/<module>/internal/*.h``
122 public/pw_foo/a_header.h
125 For headers that must be exposed due to C++ limitations (i.e. are included from
126 the public interface, but are not intended for use), place the headers in a
127 ``internal`` subfolder under the public headers directory; as
128 ``{pw_module_dir}/public/<module>/internal/*.h``. For example:
133 public/pw_foo/internal/secret.h
134 public/pw_foo/internal/business.h
138 These headers must not override headers from other modules. For
139 that, there is the ``public_overrides/`` directory.
141 C++ public override headers
142 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
143 Located ``{pw_module_dir}/public_overrides/<module>``. In general, the Pigweed
144 philosophy is to avoid having "things hiding under rocks", and having header
145 files with the same name that can override each other is considered a rock
146 where surprising things can hide. Additionally, a design goal of the Pigweed
147 module structure is to make it so there is ideally exactly one obvious place
148 to find a header based on an ``#include``.
150 However, in some cases header overrides are necessary to enable flexibly
151 combining modules. To make this as explicit as possible, headers which override
152 other headers must go in
154 ``{pw_module_dir}/public_overrides/...```
156 For example, the ``pw_unit_test`` module provides a header override for
157 ``gtest/gtest.h``. The structure of the module is (omitting some files):
163 public_overrides/gtest
164 public_overrides/gtest/gtest.h
167 public/pw_unit_test/framework.h
168 public/pw_unit_test/simple_printing_event_handler.h
169 public/pw_unit_test/event_handler.h
171 Note that the overrides are in a separate directory ``public_overrides``.
173 C++ implementation files
174 ~~~~~~~~~~~~~~~~~~~~~~~~
175 Located ``{pw_module_dir}/``. C++ implementation files go at the top level of
176 the module. Implementation files must always use "" style includes.
189 Compile-time configuration
190 ~~~~~~~~~~~~~~~~~~~~~~~~~~
191 Pigweed modules are intended to be used in a wide variety of environments.
192 In support of this, some modules expose compile-time configuration options.
193 Pigweed has an established pattern for declaring and overriding module
198 Compile-time configuration provides flexibility, but also imposes
199 restrictions. A module can only have one configuration in a given build.
200 This makes testing modules with compile-time configuration more difficult.
201 Where appropriate, consider alternatives such as C++ templates or runtime
204 Declaring configuration
205 ^^^^^^^^^^^^^^^^^^^^^^^
206 Configuration options are declared in a header file as macros. If the macro is
207 not already defined, a default definition is provided. Otherwise, nothing is
208 done. Configuration headers may include ``static_assert`` statements to validate
209 configuration values.
213 // Example configuration header
215 #ifndef PW_FOO_INPUT_BUFFER_SIZE_BYTES
216 #define PW_FOO_INPUT_BUFFER_SIZE_BYTES 128
217 #endif // PW_FOO_INPUT_BUFFER_SIZE_BYTES
219 static_assert(PW_FOO_INPUT_BUFFER_SIZE_BYTES >= 64);
221 The configuration header may go in one of three places in the module, depending
222 on whether the header should be exposed by the module or not.
228 # Publicly accessible configuration header
229 public/pw_foo/config.h
231 # Internal configuration header that is included by other module headers
232 public/pw_foo/internal/config.h
234 # Internal configuration header
235 pw_foo_private/config.h
237 The configuration header is provided by a build system library. This library
238 acts as a :ref:`facade<docs-module-structure-facades>`. The facade uses a
239 variable such as ``pw_foo_CONFIG``. In upstream Pigweed, all config facades
240 default to the ``pw_build_DEFAULT_MODULE_CONFIG`` backend. In the GN build
241 system, the config facade is declared as follows:
246 # The build target that overrides the default configuration options for this
247 # module. This should point to a source set that provides defines through a
248 # public config (which may -include a file or add defines directly).
249 pw_foo_CONFIG = pw_build_DEFAULT_MODULE_CONFIG
252 # An example source set for each potential config header location follows.
254 # Publicly accessible configuration header (most common)
255 pw_source_set("config") {
256 public = [ "public/pw_foo/config.h" ]
257 public_configs = [ ":public_include_path" ]
258 public_deps = [ pw_foo_CONFIG ]
261 # Internal configuration header that is included by other module headers
262 pw_source_set("config") {
263 sources = [ "public/pw_foo/internal/config.h" ]
264 public_configs = [ ":public_include_path" ]
265 public_deps = [ pw_foo_CONFIG ]
266 visibility = [":*"] # Only allow this module to depend on ":config"
267 friend = [":*"] # Allow this module to access the config.h header.
270 # Internal configuration header
271 pw_source_set("config") {
272 public = [ "pw_foo_private/config.h" ]
273 public_deps = [ pw_foo_CONFIG ]
274 visibility = [":*"] # Only allow this module to depend on ":config"
277 Overriding configuration
278 ^^^^^^^^^^^^^^^^^^^^^^^^
279 As noted above, all module configuration facades default to the same backend
280 (``pw_build_DEFAULT_MODULE_CONFIG``). This allows projects to override
281 configuration values for multiple modules from a single configuration backend,
282 if desired. The configuration values may also be overridden individually by
283 setting backends for the individual module configurations (e.g. in GN,
284 ``pw_foo_CONFIG = "//configuration:my_foo_config"``).
286 Configurations options are overridden by setting macros in the config backend.
287 These macro definitions can be provided through compilation options, such as
288 ``-DPW_FOO_INPUT_BUFFER_SIZE_BYTES=256``. Configuration macro definitions may
289 also be set in a header file. The header file is included using the ``-include``
292 This example shows two ways to configure a module in the GN build system.
296 # In the toolchain, set either pw_build_DEFAULT_MODULE_CONFIG or pw_foo_CONFIG
297 pw_build_DEFAULT_MODULE_CONFIG = get_path_info(":define_overrides", "abspath")
299 # This configuration sets PW_FOO_INPUT_BUFFER_SIZE_BYTES using the -D flag.
300 pw_source_set("define_overrides") {
301 public_configs = [ ":define_options" ]
304 config("define_options") {
305 defines = [ "-DPW_FOO_INPUT_BUFFER_SIZE_BYTES=256" ]
308 # This configuration sets PW_FOO_INPUT_BUFFER_SIZE_BYTES in a header file.
309 pw_source_set("include_overrides") {
310 public_configs = [ ":set_options_in_header_file" ]
312 # Header file with #define PW_FOO_INPUT_BUFFER_SIZE_BYTES 256
313 sources = [ "my_config_overrides.h" ]
316 config("set_options_in_header_file") {
319 rebase_path("my_config_overrides.h"),
323 .. admonition:: Why this config pattern is preferred
325 Alternate patterns for configuring a module include overriding the module's
326 config header or having that header optionally include a header at a known
327 path (e.g. ``pw_foo/config_overrides.h``). There are a few downsides to these
330 * The module needs its own config header that defines, provides defaults for,
331 and validates the configuration options. Replacing this header with a
332 user-defined header would require defining all options in the user's header,
333 which is cumbersome and brittle, and would bypass validation in the module's
335 * Including a config override header at a particular path would prevent
336 multiple modules from sharing the same configuration file. Multiple headers
337 could redirect to the same configuration file, but this would still require
338 creating a separate header and setting the config backend variable for each
340 * Optionally including a config override header requires boilerplate code that
341 would have to be duplicated in every configurable module.
342 * An optional config override header file would silently be excluded if the
343 file path were accidentally misspelled.
345 .. _docs-module-structure-facades:
349 In Pigweed, facades represent a dependency that can be swapped at compile time.
350 Facades are similar in concept to a virtual interface, but the implementation is
351 set by the build system. Runtime polymorphism with facades is not
352 possible, and each facade may only have one implementation (backend) per
353 toolchain compilation.
355 In the simplest sense, a facade is just a dependency represented by a variable.
356 For example, the ``pw_log`` facade is represented by the ``pw_log_BACKEND``
357 build variable. Facades typically are bundled with a build system library that
358 depends on the backend.
360 Facades are essential in some circumstances:
362 * Low-level, platform-specific features (:ref:`module-pw_cpu_exception`).
363 * Features that require a macro or non-virtual function interface
364 (:ref:`module-pw_log`, :ref:`module-pw_assert`).
365 * Highly leveraged code where a virtual interface or callback is too costly or
366 cumbersome (:ref:`module-pw_tokenizer`).
370 Modules should only use facades when necessary. Facades are permanently locked
371 to a particular implementation at compile time. Multpile backends cannot be
372 used in one build, and runtime dependency injection is not possible, which
373 makes testing difficult. Where appropriate, modules should use other
374 mechanisms, such as virtual interfaces, callbacks, or templates, in place of
377 The GN build system provides the
378 :ref:`pw_facade template<module-pw_build-facade>` as a convenient way to declare
383 Documentation should go in the root module folder, typically in the
384 ``docs.rst`` file. There must be a docgen entry for the documentation in the
385 ``BUILD.gn`` file with the target name ``docs``; so the full target for the
386 docs would be ``<module>:docs``.
390 pw_example_module/...
394 For modules with more involved documentation, create a separate directory
395 called ``docs/`` under the module root, and put the ``.rst`` files and other
396 related files (like images and diagrams) there.
400 pw_example_module/...
405 docs/image/screenshot.png
406 docs/image/diagram.svg
408 Creating a new Pigweed module
409 -----------------------------
410 To create a new Pigweed module, follow the below steps.
414 Connect with the Pigweed community (by `mailing the Pigweed list
415 <https://groups.google.com/forum/#!forum/pigweed>`_ or `raising your idea
416 in the Pigweed chat <https://discord.gg/M9NSeTA>`_) to discuss your module
417 idea before getting too far into the implementation. This can prevent
418 accidentally duplicating work, or avoiding writing code that won't get
421 1. Create module folder following `Module name`_ guidelines
422 2. Add `C++ public headers`_ files in
423 ``{pw_module_dir}/public/{pw_module_name}/``
424 3. Add `C++ implementation files`_ files in ``{pw_module_dir}/``
425 4. Add module documentation
427 - Add ``{pw_module_dir}/README.md`` that has a module summary
428 - Add ``{pw_module_dir}/docs.rst`` that contains the main module
431 5. Add build support inside of new module
433 - Add GN with ``{pw_module_dir}/BUILD.gn``
434 - Add Bazel with ``{pw_module_dir}/BUILD``
435 - Add CMake with ``{pw_module_dir}/CMakeLists.txt``
437 6. Add folder alias for new module variable in ``/modules.gni``
439 - ``dir_pw_new = get_path_info("pw_new", "abspath")``
441 7. Add new module to main GN build
443 - in ``/BUILD.gn`` to ``group("pw_modules")`` using folder alias variable
445 8. Add test target for new module in ``/BUILD.gn`` to
446 ``pw_test_group("pw_module_tests")``
447 9. Add new module to CMake build
449 - In ``/CMakeLists.txt`` add ``add_subdirectory(pw_new)``
451 10. Add the new module to docs module
453 - Add in ``docs/BUILD.gn`` to ``pw_doc_gen("docs")``
455 11. Run :ref:`module-pw_module-module-check`
457 - ``$ pw module-check {pw_module_dir}``
459 12. Contribute your module to upstream Pigweed (optional but encouraged!)