Google Git
Sign in
flutter / recipes / refs/heads/flutter-3.10-candidate.8
commit91d8939b55d05b5dfd3b51c73ef339e9362757d6[log] [tgz]
authorXilai Zhang <xilaizhang@google.com>Wed Apr 19 03:59:08 2023 +0000
committerCQ Bot Account <flutter-scoped@luci-project-accounts.iam.gserviceaccount.com>Wed Apr 19 03:59:08 2023 +0000
tree0ee9f632b9ebf73699fa381655d26dcb325b1b74
parente341b0a4e03e1bbd4b8bb412b7c669e546fbb066 [diff]
Add codesign keychain to search list.

update 04/18:
Another day of research and I finally tracked down EVERY single bit of every root causes of flakes on codesign runs. One of the major problems identified and solved was that, we now add our keychain to the search list.

The latest version has been thoroughly tested on 10+ led runs at different times, (>60 subbuilds in total?) and they all passed. I have 99.9 percent confidence that we will not see a codesign flake anymore. led runs are triggered with `led get-build 8784446248215539137 | led edit-recipe-bundle | led edit-system -p 10 | led launch`. Some example successful runs are:

https://ci.chromium.org/raw/build/logs.chromium.org/flutter/led/xilaizhang_google.com/ba79a610f55ba0d9afd53534d72bc02cd90bd8416a7f7ed7589242039a6361f2/+/build.proto?server=chromium-swarm.appspot.com

https://ci.chromium.org/raw/build/logs.chromium.org/flutter/led/xilaizhang_google.com/771e1a09798e2abfa9b548a9b1ee899573129ea67f7cbc6b0e379f2e31ea578a/+/build.proto?server=chromium-swarm.appspot.com

update 04/17:
Move all keychain setup commands into bash script to run in the same ssh session.
The passes were green in the morning, but started to turn red later in the day. One example failure was
https://ci.chromium.org/raw/build/logs.chromium.org/flutter/led/xilaizhang_google.com/b789c3878bda578f21b55983d5ec9dd6d7029e0182e60aafdfd83a385535a9a6/+/build.proto?server=chromium-swarm.appspot.com

side note, sometimes I see
"WARNING: This build ran on a canary version of LUCI. If you suspect it failed due to infra, retry the build. Next time it may use the non-canary version." but i think this might be unrelated to our use case.

Update 04/16:

I am starting to reproduce the errors in importing certificates. It appears to me that for all the successful runs, the only present certificate should be the flutter certificate. On the other hand, if the device already have two certificates
  1) 5BB6BDEBAA24047669E1A93BCF5CBD2B1E41BB67 "MDM Client Identity Certificate"
  2) CC21BF11AB9925313E54CD4FCCFEB82347E75AFA "Apple Development: Flutter Devicelab (24273VPZX7)"
then the import will be guaranteed to fail. It looks like the success depends on which bot the task is run on.


https://ci.chromium.org/raw/build/logs.chromium.org/flutter/led/xilaizhang_google.com/f8de2c9b8bc1ba038fb34202c7d4557ac5f41ceb1d1075b139f5c50cf14c0e5a/+/build.proto?server=chromium-swarm.appspot.com

https://ci.chromium.org/raw/build/logs.chromium.org/flutter/led/xilaizhang_google.com/fe251512a89251551e70c3e54ba2993e9e7eb27a874c0b806b6ebc9315fd6f6d/+/build.proto?server=chromium-swarm.appspot.com

Update 04/15:
Tuned bash script and test coverage to make them eventually pass. Learnt about set -e with if, local variable, bash expressions etc.

Was trying to test flakiness with 100 led runs and used a script to pipe luci output (not stdout or stderr) of 100 led runs using script command to google sheet. However, the 100 runs were cancelled due to contention for resources. a sample cancelled run looks like:

https://ci.chromium.org/raw/build/logs.chromium.org/flutter/led/xilaizhang_google.com/59b48935638603ac39c73aa8e8ccb69943f10423f9799026e47daae83d98736c/+/build.proto?server=chromium-swarm.appspot.com

update 4/14:
wasn't able to reproduce the flake, successful led runs are:
https://ci.chromium.org/raw/build/logs.chromium.org/flutter/led/xilaizhang_google.com/d57db265bec15c048ed82199f0d8f77a0b9e67ba109577c02d57dcd747401227/+/build.proto?server=chromium-swarm.appspot.com

https://ci.chromium.org/raw/build/logs.chromium.org/flutter/led/xilaizhang_google.com/20efcb900545754ea92434f7ba64339f3571104089dadb51e564a0a7721d7768/+/build.proto?server=chromium-swarm.appspot.com

https://ci.chromium.org/raw/build/logs.chromium.org/flutter/led/xilaizhang_google.com/4e8f7c2696f7a39ac4721198d83896f00dea821814cd3cd3918ca7895a39817b/+/build.proto?server=chromium-swarm.appspot.com

Change-Id: Ic40e62032886ca9cbe77679239fd48ba4744da04
Reviewed-on: https://flutter-review.googlesource.com/c/recipes/+/41900
Reviewed-by: Godofredo Contreras <godofredoc@google.com>
Commit-Queue: Xilai Zhang <xilaizhang@google.com>
12 files changed
tree: 0ee9f632b9ebf73699fa381655d26dcb325b1b74
  1. infra/
  2. recipe_modules/
  3. recipes/
  4. .gitignore
  5. .style.yapf
  6. AUTHORS
  7. LICENSE
  8. OWNERS
  9. README.md
  10. recipes.py
README.md

Flutter LUCI Recipes

This repository contains Flutter's LUCI recipes. For the LUCI infrastructure config, see flutter/infra. Actual builds can be seen at ci.chromium.org.

Supported repositories roll their .ci.yaml into flutter/infra, which updates what properties builds have. For example, flutter config specifies various dependencies the different tests require, which are then used by the flutter_deps recipe_module No modifications to flutter/infra are required to work on the recipes.

Configuration

Tricium configurations recipes repo.

Testing locally

Running python recipes.py test train will execute all of the tests in this repository

Troubleshooting

Required binary is not found on PATH: cipd when testing locally

You need to ensure depot_tools is installed. See Depot Tools Installation Tutorial