This Dart project contains logic for constructing infrastructure configs to validate commits in the repositories owned by Flutter.
This is the config file in a repository used to tell Cocoon what tasks are used to validate commits. It includes both the tasks used in presubmit and postsubmit.
In addition, it supports tasks from different infrastructures as long as cocoon supports that scheduler. Only luci and cocoon are supported, but contributions are welcome.
Example config:
# /.ci.yaml # Enabled branches is a list of regexes, with the assumption that these are full line matches. # Internally, Cocoon prefixes these with $ and suffixes with ^ to enable matches. enabled_branches: - main - flutter-\\d+\\.\\d+-candidate\\.\\d+ # Platform properties defines common properties shared among targets from the same platform. platform_properties: linux: properties: # os will be inherited by all Linux targets, but it can be overrided at the target level os: Linux targets: # A Target is an individual unit of work that is scheduled by Flutter infra # Target's are composed of the following properties: # name: A human readable string to uniquely identify this target. # The first word indicates the platform this test will be run on. This should match # to an existing platform under platform_properties. # recipes: LUCI recipes the target follows to run tests # https://flutter.googlesource.com/recipes/+/refs/heads/main/recipes/ # bringup: Whether this target is under active development and should not block the tree. # If true, will not run in presubmit and will not block postsubmit. # presubmit: Whether to run this target on presubmit (defaults to true). # postsubmit: Whether to run this target on postsubmit (defaults to true). # run_if: List of path regexes that can trigger this target on presubmit. # If none are passed, it will evaluare run_if_not. If both are empty the target # will always run in presubmit. # run_if_not: List of path regexes used to filter out presubmit targets. The target will # be run only if the files changed do not match any paths in this list. If run_if # is provided and not empty run_if_not will be ignored. # enabled_branches: List of strings of branches this target can run on. # This overrides the global enabled_branches. # properties: A map of string, string. Values are parsed to their closest data model. # postsubmit_properties: Properties that are only run on postsubmit. # timeout: Integer defining whole build execution time limit for all steps in minutes. # # Minimal example: # Linux analyze will run on all presubmit and in postsubmit. - name: Linux analyze # # Bringup example: # Linux licenses will run on postsubmit, but it also passes the properties # `analyze=true` to the builder. Since `bringup=true`, presubmit is not run, # and postsubmit runs will not block the tree. - name: Linux licenses bringup: true properties: - analyze: license # # Tags example: # This test will be categorized as host only framework test. # Postsubmit runs will be passed "upload_metrics: true". - name: Linux analyze properties: tags: >- ["framework", "hostonly"] postsubmit_properties: - upload_metrics: "true" # # Devicelab example: # For tests that are located https://github.com/flutter/flutter/tree/master/dev/devicelab/bin/tasks: # 1) target name follows format of `<platform> <taskname>` # 2) properties # 2.1) update `tags` based on hosts, devices, and tests type. These tags will be used for statistic analysis. # 2.2) a `taskname` property is required, which should match the task name # # Here is the target config for a task named: `analyzer_benchmark.dart`. - name: Linux_android analyzer_benchmark recipe: devicelab/devicelab_drone presubmit: false properties: tags: > ["devicelab", "android", "linux"] task_name: analyzer_benchmark
All new targets should be added as bringup: true to ensure they do not block the tree.
Targets first need to be mirrored to flutter/infra before they will be run. This propagation takes about 30 minutes, and will only run as non-blocking in postsubmit.
The target will show runs in https://ci.chromium.org/p/flutter (under the repo). See https://github.com/flutter/flutter/wiki/Adding-a-new-Test-Shard for up to date information on the steps to promote your target to blocking.
For flutter/flutter, there's a GitHub bot that will promote a test that has been passing for the past 50 runs.
This only applies to flutter/flutter
To prevent tests from rotting, all targets are required to have a clear owner. Add an owner in TESTOWNERS
Targets support specifying properties that can be passed throughout infrastructure. The following are a list of keys that are reserved for special use.
Properties is a Map<String, String> and any special values must be JSON encoded (i.e. no trailing commas). Additionally, these strings must be compatible with YAML multiline strings
$flutter/osx_sdk: xcode configs including sdk and runtime. Note: support on legacy xcode/runtime properties and xcode dependency has been deprecated.
Example:
$flutter/osx_sdk : >- { "sdk_version": "14e222b", "runtime_versions": [ "ios-16-4_14e222b", "ios-16-2_14c18" ] }
add_recipes_cq: String boolean whether to add this target to flutter/recipes CQ. This ensures changes to flutter/recipes pass on this target before landing.
dependencies: JSON list of objects with “dependency” and optionally “version”. The list of supported deps is in flutter_deps recipe_module. Dependencies generate a corresponding swarming cache that can be used in the recipe code. The path of the cache will be the name of the dependency.
Versions can be located in CIPD
Example
dependencies: >- [ {"dependency": "android_sdk"}, {"dependency": "chrome_and_driver", "version": "latest"}, {"dependency": "clang"}, {"dependency": "goldctl"} ]
tags: JSON list of strings. These are currently only used in flutter/flutter to help with TESTOWNERSHIP and test flakiness.
Example
tags: > ["devicelab","hostonly"]
test_timeout_secs String determining seconds before timeout for an individual test step. Note that this is the timeout for a single test step rather than the entire build execution timeout.
Example
test_timeout_secs: "2700"
presubmit_max_attempts The max attempts the target will be auto executed in presubmit. If it is not specified, the default value is 1 and it means no auto rerun will happen. If explicitly defined, it controls the max number of attempts. For example: 3 means it will be auto rescheduled two more times.
Example
presubmit_max_attempts: "3"
ci.yaml, find a target that would be impacted by this changeversion specified in dependencies- name: Linux Host Engine recipe: engine properties: build_host: "true" dependencies: >- [ {"dependency": "open_jdk", "version": "11"} ] timeout: 60
Note: updates on other entries except properties will not take effect immediately. Ths PR needs to be landed first to wait for changes propagated in infrastructure.
Target depends on the prefix platform in its name to decide which platform to run on. This should match to an existing platform under platform_properties.
If one target needs to switch running platforms, e.g. from a devicelab bot to a host only bot:
bringup: truebringup: falseExample: say one wants to switch Linux_android web_size__compile_test to a vm.
Existing config:
- name: Linux_android web_size__compile_test properties: tags: > ["devicelab", "android", "linux"]
Add a new config:
- name: Linux web_size__compile_test bringup: true # new target properties: dependencies: >- # optional [ {"dependency": "new-dependency", "version": "new-dependency-version"} ] tags: > ["devicelab", "hostonly", "linux"]
After validating the new target passes, lands the clean up change by removing the config of old target Linux_android web_size__compile_test and removing the bringup: true for the new target.
Note: this change may affect benchmark metrics. Notify the metrics sherrif to monitor potential regression.
Cocoon supports tests that are not owned by Flutter infrastructure. By default, these should not block the tree but act as FYI to the gardeners.
.ci.yaml# .ci.yaml # Name is an arbitrary string that will show on the build dashboard - name: my_external_test_a # External tests should not block the tree bringup: true presubmit: false # Scheduler must match what was added to scheduler.proto (any unique name works) scheduler: my_external_location
https://flutter-dashboard.appspot.com/api/update-task-status - https://github.com/flutter/cocoon/blob/master/app_dart/lib/src/request_handlers/update_task_status.dartFor targets using the Cocoon scheduler, they can run on:
By default, all targets should use the Cocoon scheduler.
bringup: true