1 # Field Trial Testing Configuration
3 This directory contains the `fieldtrial_testing_config.json` configuration file,
4 which is used to ensure test coverage of active field trials.
6 For each study, the first available experiment after platform filtering is used
7 as the default experiment for Chromium builds. This experiment is also used for
8 perf bots and various browser tests in the waterfall (e.g. browser_tests,
9 components_browsertests, content_browsertests, extensions_browsertests, interactive_ui_tests and
10 sync_integration_tests). It is not used by unit test targets.
12 > Note: This configuration applies specifically to Chromium developer builds.
13 > Chrome branded / official builds do not use these definitions by default.
14 > They can, however, be enabled with the `--enable-field-trial-config` switch.
15 > For Chrome branded Android builds, due to binary size constraints, the
16 > configuration cannot be applied by this switch.
18 > Note: Non-developer builds of Chromium (for example, non-Chrome browsers,
19 > or Chromium builds provided by Linux distros) should disable the testing
20 > config by either (1) specifying the GN flag `disable_fieldtrial_testing_config=true`,
21 > (2) specifying the `--disable-field-trial-config` switch or (3) specifying a
22 > custom variations server URL using the `--variations-server-url` switch.
24 > Note: An experiment in the testing configuration file that enables/disables a
25 > feature that is explicitly overridden (e.g. using the `--enable-features` or
26 > `--disable-features` switches) will be skipped.
34 "platforms": [Array of Strings of Valid Platforms for These Experiments],
37 "//0": "Comment Line 0. Lines 0-9 are supported.",
38 "name": "ExperimentName",
39 "params": {Dictionary of Params},
40 "enable_features": [Array of Strings of Features],
41 "disable_features": [Array of Strings of Features]
52 The config file is a dictionary at the top level mapping a study name to an
53 array of *study configurations*. The study name in the configuration file
54 **must** match the FieldTrial name used in the Chromium client code.
56 > Note: Many newer studies do not use study names in the client code at all, and
57 > rely on the [Feature List API][FeatureListAPI] instead. Nonetheless, if a
58 > study has a server-side configuration, the study `name` specified here
59 > must still match the name specified in the server-side configuration; this is
60 > used to implement consistency checks on the server.
62 ### Study Configurations
64 Each *study configuration* is a dictionary containing `platforms` and
67 `platforms` is an array of strings, indicating the targetted platforms. The
68 strings may be `android`, `android_weblayer`, `android_webview`, `chromeos`,
69 `chromeos_lacros`, `ios`, `linux`, `mac`, or `windows`.
71 `experiments` is an array containing the *experiments*.
73 The converter uses the `platforms` array to determine which experiment to use
74 for the study. The first experiment matching the active platform will be used.
76 > Note: While `experiments` is defined as an array, currently only the first
77 > entry is used*\**. We would love to be able to test all possible study
78 > configurations, but don't currently have the buildbot resources to do so.
79 > Hence, the current best-practice is to identify which experiment group is the
80 > most likely candidate for ultimate launch, and to test that configuration. If
81 > there is a server-side configuration for this study, it's typically
82 > appropriate to copy/paste one of the experiment definitions into this file.
86 > Technically, there is one exception: If there's a forcing_flag group
87 > specified in the config, that group will be used if there's a corresponding
88 > forcing_flag specified on the command line. You, dear reader, should
89 > probably not use this fancy mechanism unless you're <em>quite</em> sure you
90 > know what you're doing =)
93 ### Experiments (Groups)
94 Each *experiment* is a dictionary that must contain the `name` key, identifying
95 the experiment group name.
97 > Note: Studies should typically use the [Feature List API][FeatureListAPI]. For
98 > such studies, the experiment `name` specified in the testing config is still
99 > required (for legacy reasons), but it is ignored. However, the lists of
100 > `enable_features`, `disable_features`, and `params` **must** match the server
101 > config. This is enforced via server-side Tricorder checks.
103 > For old-school studies that do check the actual experiment group name in the
104 > client code, the `name` **must** exactly match the client code and the server
107 The remaining keys -- `enable_features`, `disable_features`, `min_os_version`,
108 and `params` -- are optional.
110 `enable_features` and `disable_features` indicate which features should be
111 enabled and disabled, respectively, through the
112 [Feature List API][FeatureListAPI].
114 `min_os_version` indicates a minimum OS version level (e.g. "10.0.0") to apply
115 the experiment. This string is decoded as a `base::Version`. The same version is
116 applied to all platforms. If you need different versions for different
117 platforms, you will need to use different studies.
119 `params` is a dictionary mapping parameter name to parameter value.
121 > Reminder: The variations framework does not actually fetch any field trial
122 > definitions from the server for Chromium builds, so any feature enabling or
123 > disabling must be configured here.
125 [FeatureListAPI]: https://cs.chromium.org/chromium/src/base/feature_list.h
129 Each experiment may have up to 10 lines of comments. The comment key must be of
130 the form `//N` where `N` is between 0 and 9.
134 "AStudyWithExperimentComment": [
136 "platforms": ["chromeos", "linux", "mac", "windows"],
139 "//0": "This is the first comment line.",
140 "//1": "This is the second comment line.",
141 "name": "DesktopExperiment"
149 ### Specifying Different Experiments for Different Platforms
150 Simply specify two different study configurations in the study:
154 "DifferentExperimentsPerPlatform": [
156 "platforms": ["chromeos", "linux", "mac", "windows"],
157 "experiments": [{ "name": "DesktopExperiment" }]
160 "platforms": ["android", "ios"],
161 "experiments": [{ "name": "MobileExperiment" }]
169 Run the following command to auto-format the `fieldtrial_testing_config.json`
173 python3 testing/variations/PRESUBMIT.py testing/variations/fieldtrial_testing_config.json
176 The presubmit tool will also ensure that your changes follow the correct